[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 21:28:34.114640 10981 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.185.126:33961
I20240502 21:28:34.116374 10981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 21:28:34.117749 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:28:34.130008 10987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.130591 10986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.131019 10989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:28:34.131649 10981 server_base.cc:1034] running on GCE node
I20240502 21:28:34.132614 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:28:34.132812 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:28:34.132903 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685314132882 us; error 0 us; skew 500 ppm
I20240502 21:28:34.133241 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:28:34.136426 10981 webserver.cc:460] Webserver started at http://127.10.185.126:41407/ using document root <none> and password file <none>
I20240502 21:28:34.137269 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:28:34.137374 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:28:34.137722 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:28:34.141485 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/master-0-root/instance:
uuid: "e91dc03c15894f9e8375117ebda800de"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.148399 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20240502 21:28:34.152179 10994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.153892 10981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20240502 21:28:34.154098 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/master-0-root
uuid: "e91dc03c15894f9e8375117ebda800de"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.154305 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:28:34.186481 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:28:34.187448 10981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 21:28:34.187754 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:28:34.209575 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.126:33961
I20240502 21:28:34.209599 11045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.126:33961 every 8 connection(s)
I20240502 21:28:34.214713 11046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.229207 11046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de: Bootstrap starting.
I20240502 21:28:34.233433 11046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.234725 11046 log.cc:824] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de: Log is configured to *not* fsync() on all Append() calls
I20240502 21:28:34.237988 11046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de: No bootstrap required, opened a new log
I20240502 21:28:34.245920 11046 raft_consensus.cc:348] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91dc03c15894f9e8375117ebda800de" member_type: VOTER }
I20240502 21:28:34.246327 11046 raft_consensus.cc:374] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.246429 11046 raft_consensus.cc:729] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91dc03c15894f9e8375117ebda800de, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.247238 11046 consensus_queue.cc:260] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91dc03c15894f9e8375117ebda800de" member_type: VOTER }
I20240502 21:28:34.247467 11046 raft_consensus.cc:388] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 21:28:34.247582 11046 raft_consensus.cc:482] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 21:28:34.247732 11046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:34.250140 11046 raft_consensus.cc:504] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91dc03c15894f9e8375117ebda800de" member_type: VOTER }
I20240502 21:28:34.251034 11046 leader_election.cc:302] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e91dc03c15894f9e8375117ebda800de; no voters:
I20240502 21:28:34.251855 11046 leader_election.cc:288] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 21:28:34.252017 11049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:34.252821 11049 raft_consensus.cc:686] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 1 LEADER]: Becoming Leader. State: Replica: e91dc03c15894f9e8375117ebda800de, State: Running, Role: LEADER
I20240502 21:28:34.253548 11049 consensus_queue.cc:237] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91dc03c15894f9e8375117ebda800de" member_type: VOTER }
I20240502 21:28:34.254372 11046 sys_catalog.cc:564] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [sys.catalog]: configured and running, proceeding with master startup.
I20240502 21:28:34.257032 11050 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e91dc03c15894f9e8375117ebda800de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91dc03c15894f9e8375117ebda800de" member_type: VOTER } }
I20240502 21:28:34.257110 11051 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91dc03c15894f9e8375117ebda800de. Latest consensus state: current_term: 1 leader_uuid: "e91dc03c15894f9e8375117ebda800de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91dc03c15894f9e8375117ebda800de" member_type: VOTER } }
I20240502 21:28:34.257381 11050 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [sys.catalog]: This master's current role is: LEADER
I20240502 21:28:34.257464 11051 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [sys.catalog]: This master's current role is: LEADER
I20240502 21:28:34.259891 11055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 21:28:34.268347 11055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 21:28:34.269887 10981 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240502 21:28:34.279760 11055 catalog_manager.cc:1347] Generated new cluster ID: 29d1be4434664fa884351d351f2ab9de
I20240502 21:28:34.279918 11055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 21:28:34.313791 11055 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 21:28:34.315677 11055 catalog_manager.cc:1504] Loading token signing keys...
I20240502 21:28:34.329144 11055 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de: Generated new TSK 0
I20240502 21:28:34.330539 11055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 21:28:34.335544 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:28:34.340909 11069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.341209 11068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.342592 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:28:34.343194 10981 server_base.cc:1034] running on GCE node
I20240502 21:28:34.343521 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:28:34.343622 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:28:34.343698 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685314343668 us; error 0 us; skew 500 ppm
I20240502 21:28:34.343910 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:28:34.345736 10981 webserver.cc:460] Webserver started at http://127.10.185.65:37699/ using document root <none> and password file <none>
I20240502 21:28:34.346031 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:28:34.346134 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:28:34.346298 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:28:34.347087 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-0-root/instance:
uuid: "d811f5b5f8d84492a2990597d41cfec8"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.350205 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240502 21:28:34.352430 11076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.352937 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240502 21:28:34.353116 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-0-root
uuid: "d811f5b5f8d84492a2990597d41cfec8"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.353302 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:28:34.367816 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:28:34.368829 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:28:34.370224 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:28:34.372340 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:28:34.372527 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.372675 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:28:34.372769 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.393568 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.65:41721
I20240502 21:28:34.393587 11139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.65:41721 every 8 connection(s)
I20240502 21:28:34.396214 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:28:34.406941 11144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.414892 11145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.422528 11147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:28:34.432761 11140 heartbeater.cc:347] Connected to a master server at 127.10.185.126:33961
I20240502 21:28:34.433207 11140 heartbeater.cc:464] Registering TS with master...
I20240502 21:28:34.434166 11140 heartbeater.cc:511] Master 127.10.185.126:33961 requested a full tablet report, sending...
I20240502 21:28:34.434841 10981 server_base.cc:1034] running on GCE node
I20240502 21:28:34.435602 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:28:34.435710 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:28:34.435796 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685314435750 us; error 0 us; skew 500 ppm
I20240502 21:28:34.436036 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:28:34.437237 11011 ts_manager.cc:194] Registered new tserver with Master: d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721)
I20240502 21:28:34.437814 10981 webserver.cc:460] Webserver started at http://127.10.185.66:38967/ using document root <none> and password file <none>
I20240502 21:28:34.438100 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:28:34.438225 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:28:34.438380 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:28:34.439201 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-1-root/instance:
uuid: "66ac0c62123043168247dd5c102d068e"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.441570 11011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40536
I20240502 21:28:34.442288 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20240502 21:28:34.444670 11152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.445153 10981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 21:28:34.445304 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-1-root
uuid: "66ac0c62123043168247dd5c102d068e"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.445487 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:28:34.454910 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:28:34.455829 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:28:34.456671 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:28:34.458070 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:28:34.458178 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.458314 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:28:34.458393 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.482663 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.66:38387
I20240502 21:28:34.482705 11216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.66:38387 every 8 connection(s)
I20240502 21:28:34.484874 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:28:34.489126 11221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:28:34.492388 11218 heartbeater.cc:347] Connected to a master server at 127.10.185.126:33961
I20240502 21:28:34.492566 11218 heartbeater.cc:464] Registering TS with master...
I20240502 21:28:34.493077 11218 heartbeater.cc:511] Master 127.10.185.126:33961 requested a full tablet report, sending...
I20240502 21:28:34.494096 11011 ts_manager.cc:194] Registered new tserver with Master: 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:34.494922 11011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40548
W20240502 21:28:34.499657 11222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:28:34.500218 11224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:28:34.500931 10981 server_base.cc:1034] running on GCE node
I20240502 21:28:34.501260 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:28:34.501349 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:28:34.501402 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685314501379 us; error 0 us; skew 500 ppm
I20240502 21:28:34.501561 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:28:34.503028 10981 webserver.cc:460] Webserver started at http://127.10.185.67:37345/ using document root <none> and password file <none>
I20240502 21:28:34.503306 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:28:34.503387 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:28:34.503518 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:28:34.504279 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-2-root/instance:
uuid: "09d0afb22b4d4843859568e9b95f80d3"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.507218 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 21:28:34.509243 11229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.509666 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20240502 21:28:34.509799 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-2-root
uuid: "09d0afb22b4d4843859568e9b95f80d3"
format_stamp: "Formatted at 2024-05-02 21:28:34 on dist-test-slave-ccs3"
I20240502 21:28:34.509928 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714685314041113-10981-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:28:34.537662 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:28:34.538383 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:28:34.539073 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:28:34.540143 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:28:34.540258 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.540377 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:28:34.540467 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:28:34.563062 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.67:35337
I20240502 21:28:34.563105 11292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.67:35337 every 8 connection(s)
I20240502 21:28:34.572911 11293 heartbeater.cc:347] Connected to a master server at 127.10.185.126:33961
I20240502 21:28:34.573149 11293 heartbeater.cc:464] Registering TS with master...
I20240502 21:28:34.573774 11293 heartbeater.cc:511] Master 127.10.185.126:33961 requested a full tablet report, sending...
I20240502 21:28:34.575085 11011 ts_manager.cc:194] Registered new tserver with Master: 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:34.575331 10981 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.010905534s
I20240502 21:28:34.576400 11011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40552
I20240502 21:28:34.577065 10981 test_util.cc:268] Using random seed: 1226027642
I20240502 21:28:34.604714 11011 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40556:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240502 21:28:34.607475 11011 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240502 21:28:34.670272 11257 tablet_service.cc:1467] Processing CreateTablet for tablet 84a1add9186b4b79b4219a7c71d098ec (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240502 21:28:34.670564 11181 tablet_service.cc:1467] Processing CreateTablet for tablet 84a1add9186b4b79b4219a7c71d098ec (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240502 21:28:34.671449 11257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a1add9186b4b79b4219a7c71d098ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.671453 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a1add9186b4b79b4219a7c71d098ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.671878 11180 tablet_service.cc:1467] Processing CreateTablet for tablet 06d771dd765f4c5b9109da7efb384c26 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240502 21:28:34.672513 11180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06d771dd765f4c5b9109da7efb384c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.673341 11255 tablet_service.cc:1467] Processing CreateTablet for tablet b3b14e88bc4f4a3d8142745b6d9aa2b6 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240502 21:28:34.674034 11255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b14e88bc4f4a3d8142745b6d9aa2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.678196 11177 tablet_service.cc:1467] Processing CreateTablet for tablet ff778a95652e4bf1907120052e3be52a (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240502 21:28:34.678876 11177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff778a95652e4bf1907120052e3be52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.684350 11175 tablet_service.cc:1467] Processing CreateTablet for tablet 046b04233e72429fa8065ddf9a5d1e2a (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240502 21:28:34.683266 11176 tablet_service.cc:1467] Processing CreateTablet for tablet f93f4a1cf91841438cd87199c63b07b5 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240502 21:28:34.690532 11174 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7e1651bcb84389ad070db5b722d0ab (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240502 21:28:34.688983 11173 tablet_service.cc:1467] Processing CreateTablet for tablet 51b42b2522314532b39caf9705afe670 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240502 21:28:34.690984 11179 tablet_service.cc:1467] Processing CreateTablet for tablet b3b14e88bc4f4a3d8142745b6d9aa2b6 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240502 21:28:34.694065 11175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 046b04233e72429fa8065ddf9a5d1e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.694851 11176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f93f4a1cf91841438cd87199c63b07b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.694799 11250 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7e1651bcb84389ad070db5b722d0ab (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240502 21:28:34.693375 11178 tablet_service.cc:1467] Processing CreateTablet for tablet cde1bc3bd44a429087a018d0b1ab0644 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240502 21:28:34.695053 11256 tablet_service.cc:1467] Processing CreateTablet for tablet 06d771dd765f4c5b9109da7efb384c26 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240502 21:28:34.698161 11254 tablet_service.cc:1467] Processing CreateTablet for tablet cde1bc3bd44a429087a018d0b1ab0644 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240502 21:28:34.694989 11253 tablet_service.cc:1467] Processing CreateTablet for tablet ff778a95652e4bf1907120052e3be52a (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240502 21:28:34.701565 11174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7e1651bcb84389ad070db5b722d0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.702327 11251 tablet_service.cc:1467] Processing CreateTablet for tablet 046b04233e72429fa8065ddf9a5d1e2a (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240502 21:28:34.699314 11252 tablet_service.cc:1467] Processing CreateTablet for tablet f93f4a1cf91841438cd87199c63b07b5 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240502 21:28:34.704558 11104 tablet_service.cc:1467] Processing CreateTablet for tablet 84a1add9186b4b79b4219a7c71d098ec (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240502 21:28:34.705763 11099 tablet_service.cc:1467] Processing CreateTablet for tablet f93f4a1cf91841438cd87199c63b07b5 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240502 21:28:34.706627 11100 tablet_service.cc:1467] Processing CreateTablet for tablet ff778a95652e4bf1907120052e3be52a (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240502 21:28:34.706724 11101 tablet_service.cc:1467] Processing CreateTablet for tablet cde1bc3bd44a429087a018d0b1ab0644 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240502 21:28:34.705554 11102 tablet_service.cc:1467] Processing CreateTablet for tablet b3b14e88bc4f4a3d8142745b6d9aa2b6 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240502 21:28:34.706776 11098 tablet_service.cc:1467] Processing CreateTablet for tablet 046b04233e72429fa8065ddf9a5d1e2a (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240502 21:28:34.707214 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7e1651bcb84389ad070db5b722d0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.708135 11173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b42b2522314532b39caf9705afe670. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.708349 11249 tablet_service.cc:1467] Processing CreateTablet for tablet 51b42b2522314532b39caf9705afe670 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240502 21:28:34.708285 11097 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7e1651bcb84389ad070db5b722d0ab (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240502 21:28:34.708252 11103 tablet_service.cc:1467] Processing CreateTablet for tablet 06d771dd765f4c5b9109da7efb384c26 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240502 21:28:34.709179 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde1bc3bd44a429087a018d0b1ab0644. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.714195 11104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a1add9186b4b79b4219a7c71d098ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.716503 11256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06d771dd765f4c5b9109da7efb384c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.717583 11102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b14e88bc4f4a3d8142745b6d9aa2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.718490 11099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f93f4a1cf91841438cd87199c63b07b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.720556 11253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff778a95652e4bf1907120052e3be52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.718520 11096 tablet_service.cc:1467] Processing CreateTablet for tablet 51b42b2522314532b39caf9705afe670 (DEFAULT_TABLE table=test-workload [id=27009336531e4366b0601a94867b9863]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240502 21:28:34.721469 11179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b14e88bc4f4a3d8142745b6d9aa2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.723037 11251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 046b04233e72429fa8065ddf9a5d1e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.726747 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff778a95652e4bf1907120052e3be52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.732152 11096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b42b2522314532b39caf9705afe670. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.734951 11254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde1bc3bd44a429087a018d0b1ab0644. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.738155 11101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde1bc3bd44a429087a018d0b1ab0644. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.744591 11252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f93f4a1cf91841438cd87199c63b07b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.747092 11103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06d771dd765f4c5b9109da7efb384c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.755296 11249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b42b2522314532b39caf9705afe670. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.760460 11098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 046b04233e72429fa8065ddf9a5d1e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.763482 11308 tablet_bootstrap.cc:492] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:34.764355 11309 tablet_bootstrap.cc:492] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:34.766762 11097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7e1651bcb84389ad070db5b722d0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:28:34.813791 11309 tablet_bootstrap.cc:654] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.833653 11308 tablet_bootstrap.cc:654] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.848438 11309 tablet_bootstrap.cc:492] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:34.848709 11308 tablet_bootstrap.cc:492] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:34.848729 11309 ts_tablet_manager.cc:1367] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.091s user 0.011s sys 0.000s
I20240502 21:28:34.848737 11310 tablet_bootstrap.cc:492] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:34.848928 11308 ts_tablet_manager.cc:1367] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.094s user 0.000s sys 0.018s
I20240502 21:28:34.849992 11308 raft_consensus.cc:348] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.850083 11309 raft_consensus.cc:348] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.850428 11308 raft_consensus.cc:374] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.850476 11309 raft_consensus.cc:374] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.850574 11308 raft_consensus.cc:729] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.850620 11309 raft_consensus.cc:729] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.850917 11308 consensus_queue.cc:260] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.850996 11309 consensus_queue.cc:260] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.851615 11310 tablet_bootstrap.cc:654] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.858834 11308 ts_tablet_manager.cc:1398] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20240502 21:28:34.859491 11218 heartbeater.cc:503] Master 127.10.185.126:33961 was elected leader, sending a full tablet report...
I20240502 21:28:34.864784 11293 heartbeater.cc:503] Master 127.10.185.126:33961 was elected leader, sending a full tablet report...
I20240502 21:28:34.866662 11309 ts_tablet_manager.cc:1398] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.018s user 0.002s sys 0.013s
I20240502 21:28:34.867142 11309 tablet_bootstrap.cc:492] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:34.869379 11309 tablet_bootstrap.cc:654] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.877681 11308 tablet_bootstrap.cc:492] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:34.883253 11308 tablet_bootstrap.cc:654] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.886377 11310 tablet_bootstrap.cc:492] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:34.886382 11309 tablet_bootstrap.cc:492] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:34.886627 11310 ts_tablet_manager.cc:1367] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.026s
I20240502 21:28:34.886809 11309 ts_tablet_manager.cc:1367] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.016s
I20240502 21:28:34.887737 11310 raft_consensus.cc:348] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.887725 11309 raft_consensus.cc:348] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:34.888079 11309 raft_consensus.cc:374] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.888197 11309 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.888563 11309 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:34.889211 11309 ts_tablet_manager.cc:1398] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:28:34.889560 11308 tablet_bootstrap.cc:492] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:34.889640 11309 tablet_bootstrap.cc:492] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:34.889775 11308 ts_tablet_manager.cc:1367] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20240502 21:28:34.890774 11308 raft_consensus.cc:348] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:34.891041 11308 raft_consensus.cc:374] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.891132 11308 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.891449 11308 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:34.891798 11309 tablet_bootstrap.cc:654] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.892104 11308 ts_tablet_manager.cc:1398] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:28:34.892493 11308 tablet_bootstrap.cc:492] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:34.894574 11308 tablet_bootstrap.cc:654] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.903604 11308 tablet_bootstrap.cc:492] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:34.903843 11308 ts_tablet_manager.cc:1367] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20240502 21:28:34.904693 11310 raft_consensus.cc:374] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.904843 11310 raft_consensus.cc:729] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.904994 11308 raft_consensus.cc:348] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.905231 11308 raft_consensus.cc:374] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.905157 11310 consensus_queue.cc:260] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.905362 11309 tablet_bootstrap.cc:492] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:34.905411 11308 raft_consensus.cc:729] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.905550 11309 ts_tablet_manager.cc:1367] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.005s
I20240502 21:28:34.905755 11308 consensus_queue.cc:260] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.906219 11310 ts_tablet_manager.cc:1398] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.019s user 0.000s sys 0.004s
I20240502 21:28:34.906308 11308 ts_tablet_manager.cc:1398] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:28:34.906476 11140 heartbeater.cc:503] Master 127.10.185.126:33961 was elected leader, sending a full tablet report...
I20240502 21:28:34.906570 11310 tablet_bootstrap.cc:492] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:34.906622 11308 tablet_bootstrap.cc:492] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:34.908414 11309 raft_consensus.cc:348] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.908843 11309 raft_consensus.cc:374] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.909049 11309 raft_consensus.cc:729] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.911269 11308 tablet_bootstrap.cc:654] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.911141 11309 consensus_queue.cc:260] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.911675 11310 tablet_bootstrap.cc:654] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.914541 11309 ts_tablet_manager.cc:1398] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20240502 21:28:34.914989 11309 tablet_bootstrap.cc:492] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:34.917234 11309 tablet_bootstrap.cc:654] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.919518 11308 tablet_bootstrap.cc:492] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:34.919829 11308 ts_tablet_manager.cc:1367] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.002s
I20240502 21:28:34.920665 11308 raft_consensus.cc:348] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.920919 11308 raft_consensus.cc:374] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.920933 11309 tablet_bootstrap.cc:492] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:34.921020 11308 raft_consensus.cc:729] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.921159 11309 ts_tablet_manager.cc:1367] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20240502 21:28:34.921358 11308 consensus_queue.cc:260] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.922010 11309 raft_consensus.cc:348] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.922245 11309 raft_consensus.cc:374] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.922336 11309 raft_consensus.cc:729] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.922389 11310 tablet_bootstrap.cc:492] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:34.922673 11310 ts_tablet_manager.cc:1367] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.004s
I20240502 21:28:34.923023 11313 raft_consensus.cc:482] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:34.922613 11309 consensus_queue.cc:260] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.923245 11313 raft_consensus.cc:504] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.923867 11310 raft_consensus.cc:348] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.924150 11310 raft_consensus.cc:374] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.924263 11310 raft_consensus.cc:729] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.924553 11310 consensus_queue.cc:260] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.924728 11308 ts_tablet_manager.cc:1398] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:28:34.925149 11310 ts_tablet_manager.cc:1398] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:28:34.925503 11313 leader_election.cc:288] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:34.926333 11310 tablet_bootstrap.cc:492] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:34.928534 11310 tablet_bootstrap.cc:654] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.934937 11308 tablet_bootstrap.cc:492] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:34.937880 11308 tablet_bootstrap.cc:654] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.939023 11313 raft_consensus.cc:482] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:34.939301 11313 raft_consensus.cc:504] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.940163 11313 leader_election.cc:288] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:34.943187 11309 ts_tablet_manager.cc:1398] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20240502 21:28:34.943631 11309 tablet_bootstrap.cc:492] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:34.945804 11309 tablet_bootstrap.cc:654] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.947816 11310 tablet_bootstrap.cc:492] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:34.948066 11310 ts_tablet_manager.cc:1367] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.003s
I20240502 21:28:34.949321 11310 raft_consensus.cc:348] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.949610 11310 raft_consensus.cc:374] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.949770 11310 raft_consensus.cc:729] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.951512 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:34.960539 11313 raft_consensus.cc:482] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:34.961097 11310 consensus_queue.cc:260] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.951372 11268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
I20240502 21:28:34.962064 11313 raft_consensus.cc:504] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.962355 11310 ts_tablet_manager.cc:1398] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20240502 21:28:34.961601 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:34.951539 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
I20240502 21:28:34.964916 11313 leader_election.cc:288] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
W20240502 21:28:34.965480 11154 leader_election.cc:341] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:34.966514 11310 tablet_bootstrap.cc:492] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:34.966634 11309 tablet_bootstrap.cc:492] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
W20240502 21:28:34.966610 11154 leader_election.cc:341] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:34.966835 11309 ts_tablet_manager.cc:1367] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.011s
I20240502 21:28:34.967181 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:34.967667 11268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
W20240502 21:28:34.968124 11154 leader_election.cc:341] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:34.968456 11308 tablet_bootstrap.cc:492] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:34.968449 11154 leader_election.cc:302] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66ac0c62123043168247dd5c102d068e; no voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:34.968698 11309 raft_consensus.cc:348] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.968842 11308 ts_tablet_manager.cc:1367] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.000s
I20240502 21:28:34.968652 11268 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 0.
I20240502 21:28:34.969022 11309 raft_consensus.cc:374] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.969166 11309 raft_consensus.cc:729] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.969623 11313 raft_consensus.cc:2726] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:28:34.969475 11309 consensus_queue.cc:260] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.969842 11308 raft_consensus.cc:348] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.970103 11308 raft_consensus.cc:374] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.970219 11308 raft_consensus.cc:729] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
W20240502 21:28:34.970599 11154 leader_election.cc:341] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:34.970547 11308 consensus_queue.cc:260] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.970989 11154 leader_election.cc:302] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66ac0c62123043168247dd5c102d068e; no voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:34.971325 11313 raft_consensus.cc:2726] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:28:34.971848 11308 ts_tablet_manager.cc:1398] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:28:34.971879 11154 leader_election.cc:302] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:34.972251 11308 tablet_bootstrap.cc:492] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:34.972311 11313 raft_consensus.cc:2781] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:34.972440 11313 raft_consensus.cc:482] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:34.972558 11313 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:34.973049 11309 ts_tablet_manager.cc:1398] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 21:28:34.974491 11308 tablet_bootstrap.cc:654] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.976219 11309 tablet_bootstrap.cc:492] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:34.976433 11313 raft_consensus.cc:504] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.977030 11310 tablet_bootstrap.cc:654] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.977488 11313 leader_election.cc:288] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
W20240502 21:28:34.977865 11154 leader_election.cc:341] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:34.978391 11309 tablet_bootstrap.cc:654] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:34.981062 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:34.981395 11268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:34.981698 11268 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:28:34.984723 11219 tablet.cc:2369] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 21:28:34.986171 11268 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 1.
I20240502 21:28:34.986763 11154 leader_election.cc:302] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:34.986899 11309 tablet_bootstrap.cc:492] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:34.987144 11309 ts_tablet_manager.cc:1367] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20240502 21:28:34.987394 11313 raft_consensus.cc:2781] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election won for term 1
W20240502 21:28:34.987762 11154 leader_election.cc:341] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:34.988134 11309 raft_consensus.cc:348] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.988391 11309 raft_consensus.cc:374] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:34.988503 11309 raft_consensus.cc:729] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:34.988818 11309 consensus_queue.cc:260] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.991307 11321 raft_consensus.cc:482] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:34.991506 11321 raft_consensus.cc:504] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:34.993649 11321 leader_election.cc:288] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:34.999279 11308 tablet_bootstrap.cc:492] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:34.999526 11308 ts_tablet_manager.cc:1367] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.000s
I20240502 21:28:35.000567 11308 raft_consensus.cc:348] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.000859 11308 raft_consensus.cc:374] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.001005 11308 raft_consensus.cc:729] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.001901 11310 tablet_bootstrap.cc:492] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:35.002115 11310 ts_tablet_manager.cc:1367] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.005s
I20240502 21:28:35.002971 11310 raft_consensus.cc:348] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.003250 11310 raft_consensus.cc:374] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.003346 11310 raft_consensus.cc:729] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.003671 11310 consensus_queue.cc:260] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.004305 11310 ts_tablet_manager.cc:1398] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:28:35.004670 11310 tablet_bootstrap.cc:492] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:35.006557 11310 tablet_bootstrap.cc:654] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.007942 11308 consensus_queue.cc:260] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.011158 11309 ts_tablet_manager.cc:1398] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20240502 21:28:35.011327 11308 ts_tablet_manager.cc:1398] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.012s user 0.003s sys 0.001s
I20240502 21:28:35.011766 11308 tablet_bootstrap.cc:492] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:35.013865 11308 tablet_bootstrap.cc:654] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.015748 11309 tablet_bootstrap.cc:492] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:35.016475 11310 tablet_bootstrap.cc:492] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:35.016676 11310 ts_tablet_manager.cc:1367] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.002s
I20240502 21:28:35.021764 11309 tablet_bootstrap.cc:654] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.023290 11310 raft_consensus.cc:348] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.024045 11310 raft_consensus.cc:374] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.024356 11310 raft_consensus.cc:729] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.027511 11310 consensus_queue.cc:260] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.028838 11310 ts_tablet_manager.cc:1398] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20240502 21:28:35.029321 11310 tablet_bootstrap.cc:492] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:35.032052 11310 tablet_bootstrap.cc:654] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.036739 11308 tablet_bootstrap.cc:492] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:35.037014 11308 ts_tablet_manager.cc:1367] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.007s
I20240502 21:28:35.039795 11308 raft_consensus.cc:348] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.040184 11308 raft_consensus.cc:374] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.040450 11308 raft_consensus.cc:729] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.043509 11313 raft_consensus.cc:686] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.043061 11308 consensus_queue.cc:260] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.044191 11313 consensus_queue.cc:237] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.045289 11309 tablet_bootstrap.cc:492] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:35.045568 11309 ts_tablet_manager.cc:1367] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.012s
I20240502 21:28:35.046057 11308 ts_tablet_manager.cc:1398] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20240502 21:28:35.046519 11308 tablet_bootstrap.cc:492] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:35.050360 11309 raft_consensus.cc:348] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.051476 11309 raft_consensus.cc:374] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.051816 11310 tablet_bootstrap.cc:492] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:35.052582 11309 raft_consensus.cc:729] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.053086 11310 ts_tablet_manager.cc:1367] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.002s
I20240502 21:28:35.054159 11310 raft_consensus.cc:348] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.055433 11309 consensus_queue.cc:260] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.057384 11310 raft_consensus.cc:374] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.058122 11310 raft_consensus.cc:729] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.059998 11308 tablet_bootstrap.cc:654] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.061187 11309 ts_tablet_manager.cc:1398] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.012s user 0.002s sys 0.002s
I20240502 21:28:35.061692 11309 tablet_bootstrap.cc:492] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:35.062311 11011 catalog_manager.cc:5484] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 1 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.065464 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.065858 11192 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 0.
I20240502 21:28:35.066428 11309 tablet_bootstrap.cc:654] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.066744 11077 leader_election.cc:302] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.067008 11313 raft_consensus.cc:482] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.067498 11321 raft_consensus.cc:2781] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.067436 11313 raft_consensus.cc:504] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.067664 11321 raft_consensus.cc:482] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.068001 11321 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.068914 11268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
I20240502 21:28:35.069149 11268 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 0.
I20240502 21:28:35.069669 11154 leader_election.cc:302] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:35.070129 11332 raft_consensus.cc:2781] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.070273 11332 raft_consensus.cc:482] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.070444 11332 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.071388 11321 raft_consensus.cc:504] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.070279 11268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
I20240502 21:28:35.063485 11310 consensus_queue.cc:260] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.077170 11321 leader_election.cc:288] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.078550 11268 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 0.
I20240502 21:28:35.078374 11313 leader_election.cc:288] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337), d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721)
I20240502 21:28:35.079563 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.079633 11314 raft_consensus.cc:482] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.079901 11192 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.079936 11314 raft_consensus.cc:504] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.081305 11314 leader_election.cc:288] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.083562 11314 raft_consensus.cc:482] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.083828 11314 raft_consensus.cc:504] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.085027 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:35.085491 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.092588 11267 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:28:35.094772 11154 leader_election.cc:341] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:35.095400 11314 leader_election.cc:288] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.098289 11332 raft_consensus.cc:504] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.098853 11267 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 1.
I20240502 21:28:35.099328 11192 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 1.
I20240502 21:28:35.099743 11078 leader_election.cc:302] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.100256 11321 raft_consensus.cc:2781] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.100461 11321 raft_consensus.cc:686] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:35.100898 11321 consensus_queue.cc:237] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.103016 11332 leader_election.cc:288] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Requested vote from peers 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337), d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721)
I20240502 21:28:35.103722 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
W20240502 21:28:35.104931 11154 leader_election.cc:341] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:35.105438 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.105788 11267 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.110352 11267 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 1.
I20240502 21:28:35.111289 11154 leader_election.cc:302] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters: d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:35.112126 11332 raft_consensus.cc:2781] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.112615 11332 raft_consensus.cc:686] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.114528 11337 raft_consensus.cc:482] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.114279 11332 consensus_queue.cc:237] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.115983 11310 ts_tablet_manager.cc:1398] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.061s user 0.001s sys 0.006s
I20240502 21:28:35.114881 11337 raft_consensus.cc:504] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.121245 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b42b2522314532b39caf9705afe670" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:35.123384 11310 tablet_bootstrap.cc:492] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:35.123378 11114 raft_consensus.cc:2454] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 0.
I20240502 21:28:35.126679 11154 leader_election.cc:302] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.126652 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b42b2522314532b39caf9705afe670" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
W20240502 21:28:35.127727 11154 leader_election.cc:341] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:35.128315 11309 tablet_bootstrap.cc:492] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:35.129124 11337 leader_election.cc:288] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.130075 11313 raft_consensus.cc:2781] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.130086 11308 tablet_bootstrap.cc:492] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:35.130241 11313 raft_consensus.cc:482] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.121129 11009 catalog_manager.cc:5484] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.130749 11309 ts_tablet_manager.cc:1367] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.068s user 0.017s sys 0.025s
I20240502 21:28:35.132259 11313 raft_consensus.cc:3037] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.134581 11308 ts_tablet_manager.cc:1367] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.084s user 0.035s sys 0.016s
I20240502 21:28:35.135679 11308 raft_consensus.cc:348] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.136174 11308 raft_consensus.cc:374] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.136303 11308 raft_consensus.cc:729] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.136617 11308 consensus_queue.cc:260] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.137197 11308 ts_tablet_manager.cc:1398] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 21:28:35.137444 11310 tablet_bootstrap.cc:654] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.137559 11308 tablet_bootstrap.cc:492] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e: Bootstrap starting.
I20240502 21:28:35.138936 11321 raft_consensus.cc:482] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.139751 11321 raft_consensus.cc:504] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.141551 11321 leader_election.cc:288] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.144215 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
I20240502 21:28:35.143551 11191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.144701 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.145928 11309 raft_consensus.cc:348] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.147050 11267 raft_consensus.cc:2379] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d811f5b5f8d84492a2990597d41cfec8 in current term 1: Already voted for candidate 66ac0c62123043168247dd5c102d068e in this term.
I20240502 21:28:35.143903 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.148922 11309 raft_consensus.cc:374] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.149114 11309 raft_consensus.cc:729] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.149166 11078 leader_election.cc:302] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d811f5b5f8d84492a2990597d41cfec8; no voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e
I20240502 21:28:35.150338 11190 raft_consensus.cc:2454] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 0.
I20240502 21:28:35.150333 11009 catalog_manager.cc:5484] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 1 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } }
W20240502 21:28:35.151878 11230 leader_election.cc:341] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:28:35.152598 11321 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.152623 11309 consensus_queue.cc:260] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.154399 11308 tablet_bootstrap.cc:654] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.153323 11230 leader_election.cc:302] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:35.154748 11309 ts_tablet_manager.cc:1398] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.024s user 0.002s sys 0.002s
I20240502 21:28:35.157131 11309 tablet_bootstrap.cc:492] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3: Bootstrap starting.
I20240502 21:28:35.157371 11314 raft_consensus.cc:2781] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.157498 11314 raft_consensus.cc:482] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.157895 11314 raft_consensus.cc:3037] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.162220 11313 raft_consensus.cc:504] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.163681 11321 raft_consensus.cc:2726] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:28:35.166174 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b42b2522314532b39caf9705afe670" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.166868 11310 tablet_bootstrap.cc:492] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:35.167335 11314 raft_consensus.cc:504] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.168156 11310 ts_tablet_manager.cc:1367] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.050s user 0.010s sys 0.006s
I20240502 21:28:35.168184 11313 leader_election.cc:288] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.168474 11114 raft_consensus.cc:3037] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.168520 11314 leader_election.cc:288] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.170722 11308 tablet_bootstrap.cc:492] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e: No bootstrap required, opened a new log
I20240502 21:28:35.170864 11308 ts_tablet_manager.cc:1367] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.000s
I20240502 21:28:35.171259 11114 raft_consensus.cc:2454] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 1.
I20240502 21:28:35.172078 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b42b2522314532b39caf9705afe670" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.172415 11154 leader_election.cc:302] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.172665 11308 raft_consensus.cc:348] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.172996 11313 raft_consensus.cc:2781] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.173007 11267 raft_consensus.cc:3037] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.173018 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.173884 11190 raft_consensus.cc:3037] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.173907 11313 raft_consensus.cc:686] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.174362 11308 raft_consensus.cc:374] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.174525 11313 consensus_queue.cc:237] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.178954 11308 raft_consensus.cc:729] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.180222 11310 raft_consensus.cc:348] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.181661 11309 tablet_bootstrap.cc:654] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.181615 11308 consensus_queue.cc:260] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.182166 11310 raft_consensus.cc:374] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.188628 11308 ts_tablet_manager.cc:1398] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20240502 21:28:35.189000 11009 catalog_manager.cc:5484] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 1 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.189502 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.189639 11114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:35.189795 11112 raft_consensus.cc:3037] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.189865 11114 raft_consensus.cc:2454] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 0.
I20240502 21:28:35.190229 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:35.189324 11190 raft_consensus.cc:2454] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.191068 11230 leader_election.cc:302] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:35.191152 11267 raft_consensus.cc:2454] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 1.
I20240502 21:28:35.191097 11231 leader_election.cc:302] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters: 66ac0c62123043168247dd5c102d068e
I20240502 21:28:35.191876 11310 raft_consensus.cc:729] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.192190 11310 consensus_queue.cc:260] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.192835 11310 ts_tablet_manager.cc:1398] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.024s user 0.002s sys 0.000s
I20240502 21:28:35.193006 11112 raft_consensus.cc:2454] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.193112 11314 raft_consensus.cc:2781] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.195111 11310 tablet_bootstrap.cc:492] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:35.196398 11314 raft_consensus.cc:686] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.196758 11314 consensus_queue.cc:237] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.193934 11309 tablet_bootstrap.cc:492] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3: No bootstrap required, opened a new log
I20240502 21:28:35.198344 11309 ts_tablet_manager.cc:1367] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.003s
I20240502 21:28:35.200197 11310 tablet_bootstrap.cc:654] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.200528 11309 raft_consensus.cc:348] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.200464 11011 catalog_manager.cc:5484] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:35.200865 11309 raft_consensus.cc:374] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.200987 11309 raft_consensus.cc:729] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.201258 11309 consensus_queue.cc:260] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.202025 11309 ts_tablet_manager.cc:1398] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:28:35.194314 11353 raft_consensus.cc:2781] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.202467 11353 raft_consensus.cc:482] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.202764 11353 raft_consensus.cc:3037] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.206471 11321 raft_consensus.cc:482] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.206666 11321 raft_consensus.cc:504] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.207490 11321 leader_election.cc:288] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.207819 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.208191 11190 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 0.
I20240502 21:28:35.207901 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3" is_pre_election: true
I20240502 21:28:35.208621 11077 leader_election.cc:302] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.208673 11267 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 0.
I20240502 21:28:35.208928 11321 raft_consensus.cc:2781] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.209059 11321 raft_consensus.cc:482] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.209170 11321 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.209345 11353 raft_consensus.cc:504] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.210598 11310 tablet_bootstrap.cc:492] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:35.210834 11310 ts_tablet_manager.cc:1367] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.004s
I20240502 21:28:35.210759 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.211009 11112 raft_consensus.cc:3037] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.211887 11310 raft_consensus.cc:348] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.212172 11310 raft_consensus.cc:374] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.212261 11310 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.212693 11321 raft_consensus.cc:504] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.212702 11310 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.213575 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.213734 11310 ts_tablet_manager.cc:1398] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 21:28:35.214134 11190 raft_consensus.cc:3037] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.214289 11321 leader_election.cc:288] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.214509 11112 raft_consensus.cc:2454] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.215449 11231 leader_election.cc:302] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.215780 11314 raft_consensus.cc:2781] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.215991 11314 raft_consensus.cc:686] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.216327 11314 consensus_queue.cc:237] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.217563 11190 raft_consensus.cc:2454] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.214629 11353 leader_election.cc:288] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.214704 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.218562 11192 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.214907 11310 tablet_bootstrap.cc:492] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8: Bootstrap starting.
I20240502 21:28:35.214936 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.220525 11267 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.222245 11192 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 1.
I20240502 21:28:35.222838 11077 leader_election.cc:302] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.223166 11321 raft_consensus.cc:2781] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.223328 11321 raft_consensus.cc:686] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:35.223870 11321 consensus_queue.cc:237] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.224565 11267 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 1.
I20240502 21:28:35.224931 11009 catalog_manager.cc:5484] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:35.225679 11310 tablet_bootstrap.cc:654] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8: Neither blocks nor log segments found. Creating new log.
I20240502 21:28:35.228032 11310 tablet_bootstrap.cc:492] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8: No bootstrap required, opened a new log
I20240502 21:28:35.228282 11310 ts_tablet_manager.cc:1367] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.001s
I20240502 21:28:35.229274 11310 raft_consensus.cc:348] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.229591 11310 raft_consensus.cc:374] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:28:35.229710 11310 raft_consensus.cc:729] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Initialized, Role: FOLLOWER
I20240502 21:28:35.229990 11310 consensus_queue.cc:260] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.230422 11353 raft_consensus.cc:482] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.230713 11310 ts_tablet_manager.cc:1398] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 21:28:35.230687 11353 raft_consensus.cc:504] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.231909 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.232175 11192 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 0.
I20240502 21:28:35.232547 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:35.232654 11230 leader_election.cc:302] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:35.232757 11112 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 0.
I20240502 21:28:35.233207 11314 raft_consensus.cc:2781] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.233358 11314 raft_consensus.cc:482] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.233476 11314 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.235662 11314 raft_consensus.cc:504] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.236491 11314 leader_election.cc:288] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.236676 11353 leader_election.cc:288] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.236892 11314 raft_consensus.cc:482] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:28:35.237085 11314 raft_consensus.cc:504] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.237787 11314 leader_election.cc:288] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.238394 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8" is_pre_election: true
I20240502 21:28:35.238250 11009 catalog_manager.cc:5484] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.238636 11113 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 0.
I20240502 21:28:35.238729 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e" is_pre_election: true
I20240502 21:28:35.238932 11190 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 0.
I20240502 21:28:35.239322 11231 leader_election.cc:302] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.239886 11353 raft_consensus.cc:2781] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:28:35.240175 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.240494 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.240782 11112 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.243796 11112 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.240784 11192 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.244266 11231 leader_election.cc:302] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.244652 11314 raft_consensus.cc:2781] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.245185 11314 raft_consensus.cc:686] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.245533 11314 consensus_queue.cc:237] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.247242 11192 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.240658 11353 raft_consensus.cc:482] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:28:35.248078 11353 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.248891 11009 catalog_manager.cc:5484] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:35.251652 11353 raft_consensus.cc:504] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.252470 11353 leader_election.cc:288] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.252894 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.253015 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.253216 11112 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.253213 11192 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.256505 11112 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.256505 11192 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 1.
I20240502 21:28:35.257035 11230 leader_election.cc:302] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:35.257408 11353 raft_consensus.cc:2781] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:28:35.257593 11353 raft_consensus.cc:686] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.257901 11353 consensus_queue.cc:237] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.261109 11009 catalog_manager.cc:5484] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:35.283934 11192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:39070
I20240502 21:28:35.284337 11192 raft_consensus.cc:595] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Received request to transfer leadership to TS d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:35.287061 11192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:39070
I20240502 21:28:35.287292 11192 raft_consensus.cc:595] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Received request to transfer leadership to TS d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:35.296573 11267 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:32830
I20240502 21:28:35.296811 11267 raft_consensus.cc:595] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:35.299280 11267 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:32830
I20240502 21:28:35.299496 11267 raft_consensus.cc:595] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:35.301797 11267 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:32830
I20240502 21:28:35.301990 11267 raft_consensus.cc:595] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:35.304203 11267 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:32830
I20240502 21:28:35.304414 11267 raft_consensus.cc:595] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:35.306871 11192 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "51b42b2522314532b39caf9705afe670"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:39070
I20240502 21:28:35.307117 11192 raft_consensus.cc:595] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Received request to transfer leadership to TS 09d0afb22b4d4843859568e9b95f80d3
I20240502 21:28:35.430989 11313 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.434707 11332 raft_consensus.cc:979] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: : Instructing follower d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:35.434967 11332 raft_consensus.cc:1067] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Signalling peer d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:35.436172 11113 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:36318
I20240502 21:28:35.437634 11113 raft_consensus.cc:482] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.437798 11113 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.439323 11332 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.440322 11113 raft_consensus.cc:504] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.441920 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.442188 11192 raft_consensus.cc:3032] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.442283 11192 raft_consensus.cc:729] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.442797 11113 leader_election.cc:288] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 2 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.442660 11192 consensus_queue.cc:260] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.443033 11192 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.444150 11268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.446228 11192 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 2.
I20240502 21:28:35.446627 11077 leader_election.cc:302] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters: 09d0afb22b4d4843859568e9b95f80d3
I20240502 21:28:35.447129 11321 raft_consensus.cc:2781] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.447362 11321 raft_consensus.cc:686] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:35.447700 11321 consensus_queue.cc:237] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.450219 11009 catalog_manager.cc:5484] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 1 to 2, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 2 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.485138 11321 consensus_queue.cc:1035] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.490281 11369 consensus_queue.cc:1035] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.556866 11113 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:28:35.559747 11332 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.562489 11337 raft_consensus.cc:979] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: : Instructing follower d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:35.562682 11313 raft_consensus.cc:1067] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Signalling peer d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:35.563328 11112 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:36318
I20240502 21:28:35.563568 11112 raft_consensus.cc:482] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.563726 11112 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.565387 11313 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.566828 11112 raft_consensus.cc:504] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.568351 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.568624 11192 raft_consensus.cc:3032] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.568715 11192 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.569008 11192 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.569434 11192 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.571246 11112 leader_election.cc:288] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 2 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.571940 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.572203 11267 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.572289 11192 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 2.
I20240502 21:28:35.572700 11077 leader_election.cc:302] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.573235 11369 raft_consensus.cc:2781] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.573463 11369 raft_consensus.cc:686] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:35.573817 11369 consensus_queue.cc:237] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:35.575281 11267 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 2.
I20240502 21:28:35.576372 11011 catalog_manager.cc:5484] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 1 to 2, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 2 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:35.610766 11353 consensus_queue.cc:1035] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.614194 11314 raft_consensus.cc:979] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.614408 11314 raft_consensus.cc:1067] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.615427 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39058
I20240502 21:28:35.616075 11190 raft_consensus.cc:482] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.616248 11190 raft_consensus.cc:3037] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.617064 11314 consensus_queue.cc:1035] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.619127 11190 raft_consensus.cc:504] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.619937 11190 leader_election.cc:288] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.620357 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.620632 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e1651bcb84389ad070db5b722d0ab" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.620689 11112 raft_consensus.cc:3037] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.620869 11267 raft_consensus.cc:3032] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.621753 11267 raft_consensus.cc:729] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.622129 11267 consensus_queue.cc:260] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.622619 11267 raft_consensus.cc:3037] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.623433 11112 raft_consensus.cc:2454] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.624361 11154 leader_election.cc:302] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.624696 11313 raft_consensus.cc:2781] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.624897 11313 raft_consensus.cc:686] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.625025 11267 raft_consensus.cc:2454] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.625231 11313 consensus_queue.cc:237] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.627885 11009 catalog_manager.cc:5484] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 1 to 2, leader changed from 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 2 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.630170 11314 consensus_queue.cc:1035] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.635133 11354 consensus_queue.cc:1035] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.637940 11353 raft_consensus.cc:979] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.638149 11353 raft_consensus.cc:1067] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.638778 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39058
I20240502 21:28:35.639024 11190 raft_consensus.cc:482] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.639158 11190 raft_consensus.cc:3037] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.641173 11190 raft_consensus.cc:504] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.641870 11190 leader_election.cc:288] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.642447 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.642496 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93f4a1cf91841438cd87199c63b07b5" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.642647 11267 raft_consensus.cc:3032] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.642733 11112 raft_consensus.cc:3037] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.642750 11267 raft_consensus.cc:729] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.643105 11267 consensus_queue.cc:260] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.643571 11267 raft_consensus.cc:3037] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.646062 11112 raft_consensus.cc:2454] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.646521 11154 leader_election.cc:302] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.646567 11267 raft_consensus.cc:2454] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.646862 11337 raft_consensus.cc:2781] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.647022 11337 raft_consensus.cc:686] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.647320 11337 consensus_queue.cc:237] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.649422 11009 catalog_manager.cc:5484] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 1 to 2, leader changed from 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 2 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.663856 11353 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.668534 11353 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.671851 11354 raft_consensus.cc:979] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.672107 11354 raft_consensus.cc:1067] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.672812 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39058
I20240502 21:28:35.673058 11190 raft_consensus.cc:482] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.673197 11190 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.675266 11190 raft_consensus.cc:504] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.676016 11190 leader_election.cc:288] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.676527 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.676568 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.676836 11267 raft_consensus.cc:3032] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.676877 11112 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.676985 11267 raft_consensus.cc:729] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.677327 11267 consensus_queue.cc:260] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.677739 11267 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.679631 11112 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.680014 11267 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.680117 11154 leader_election.cc:302] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.680438 11337 raft_consensus.cc:2781] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.680641 11337 raft_consensus.cc:686] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.680979 11337 consensus_queue.cc:237] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.683413 11011 catalog_manager.cc:5484] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 1 to 2, leader changed from 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 2 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.689254 11337 consensus_queue.cc:1035] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.694027 11332 consensus_queue.cc:1035] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.697264 11313 raft_consensus.cc:979] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: : Instructing follower 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:35.697466 11313 raft_consensus.cc:1067] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Signalling peer 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:35.698153 11267 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "51b42b2522314532b39caf9705afe670"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:32822
I20240502 21:28:35.698385 11267 raft_consensus.cc:482] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.698482 11267 raft_consensus.cc:3037] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.701402 11267 raft_consensus.cc:504] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.702016 11369 consensus_queue.cc:1035] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.702168 11267 leader_election.cc:288] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:35.702531 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b42b2522314532b39caf9705afe670" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.702780 11112 raft_consensus.cc:3037] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.703037 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b42b2522314532b39caf9705afe670" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:35.703241 11190 raft_consensus.cc:3032] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.703332 11190 raft_consensus.cc:729] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.703657 11190 consensus_queue.cc:260] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.704092 11190 raft_consensus.cc:3037] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.706211 11112 raft_consensus.cc:2454] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 2.
I20240502 21:28:35.706943 11231 leader_election.cc:302] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:35.707139 11190 raft_consensus.cc:2454] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 2.
I20240502 21:28:35.707331 11354 raft_consensus.cc:2781] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.707504 11354 raft_consensus.cc:686] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.707825 11354 consensus_queue.cc:237] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.710673 11321 consensus_queue.cc:1035] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.711145 11011 catalog_manager.cc:5484] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 1 to 2, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 2 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:35.840236 11354 consensus_queue.cc:1035] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.843055 11353 raft_consensus.cc:979] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.843271 11353 raft_consensus.cc:1067] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:35.844130 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39058
I20240502 21:28:35.845032 11190 raft_consensus.cc:482] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:35.845188 11190 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.845510 11354 consensus_queue.cc:1035] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:28:35.848136 11190 raft_consensus.cc:504] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.848896 11190 leader_election.cc:288] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:35.849306 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:35.849462 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:35.849582 11112 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.850570 11267 raft_consensus.cc:3032] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:35.850694 11267 raft_consensus.cc:729] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:35.850980 11267 consensus_queue.cc:260] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.851400 11267 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.853654 11112 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.854246 11267 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:35.854679 11154 leader_election.cc:302] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:35.855016 11313 raft_consensus.cc:2781] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:35.855185 11313 raft_consensus.cc:686] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:35.855530 11313 consensus_queue.cc:237] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:35.857933 11011 catalog_manager.cc:5484] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 1 to 2, leader changed from 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 2 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:35.970599 11267 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:35.972978 11267 raft_consensus.cc:1261] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:35.973652 11321 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:35.977393 11190 raft_consensus.cc:1261] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:35.977934 11369 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.014297 11190 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.014991 11369 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.018319 11267 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.019080 11321 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.090004 11267 raft_consensus.cc:1261] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.090670 11313 consensus_queue.cc:1035] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.094285 11112 raft_consensus.cc:1261] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.095079 11337 consensus_queue.cc:1035] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.096913 11190 raft_consensus.cc:1261] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.098196 11354 consensus_queue.cc:1035] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.101459 11112 raft_consensus.cc:1261] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.102133 11353 consensus_queue.cc:1035] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.136684 11267 raft_consensus.cc:1261] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.137312 11337 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.140467 11112 raft_consensus.cc:1261] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.141036 11337 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.147053 11112 raft_consensus.cc:1261] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.147687 11337 consensus_queue.cc:1035] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.151479 11267 raft_consensus.cc:1261] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.152629 11337 consensus_queue.cc:1035] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.341490 11267 raft_consensus.cc:1261] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.342121 11337 consensus_queue.cc:1035] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:36.345664 11112 raft_consensus.cc:1261] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:36.346899 11337 consensus_queue.cc:1035] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:38.324527 10981 auto_leader_rebalancer-test.cc:261] The leader distribution is
I20240502 21:28:38.324935 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:38.333617 11112 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
mode: GRACEFUL
new_leader_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:52156
I20240502 21:28:38.333859 11112 raft_consensus.cc:595] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Received request to transfer leadership to TS 09d0afb22b4d4843859568e9b95f80d3
I20240502 21:28:38.340737 11190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:52416
I20240502 21:28:38.340987 11190 raft_consensus.cc:595] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Received request to transfer leadership to TS 09d0afb22b4d4843859568e9b95f80d3
I20240502 21:28:38.341572 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240502 21:28:38.341874 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:38.699764 11389 raft_consensus.cc:979] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: : Instructing follower 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:38.700023 11389 raft_consensus.cc:1067] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Signalling peer 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:38.700899 11267 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:32822
I20240502 21:28:38.701160 11267 raft_consensus.cc:482] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:38.701289 11267 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:38.703394 11267 raft_consensus.cc:504] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:38.704138 11267 leader_election.cc:288] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 3 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:38.704818 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:38.704818 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:38.705135 11190 raft_consensus.cc:3032] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:28:38.705199 11112 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:38.705245 11190 raft_consensus.cc:729] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:38.705598 11190 consensus_queue.cc:260] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:38.706033 11190 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:38.708240 11112 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 3.
I20240502 21:28:38.708757 11231 leader_election.cc:302] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:38.708914 11190 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 3.
I20240502 21:28:38.709064 11385 raft_consensus.cc:2781] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:28:38.709604 11385 raft_consensus.cc:686] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:38.709986 11385 consensus_queue.cc:237] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:38.713218 11010 catalog_manager.cc:5484] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 2 to 3, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 3 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:38.794286 11346 raft_consensus.cc:979] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: : Instructing follower 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:38.794490 11346 raft_consensus.cc:1067] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Signalling peer 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:38.795279 11267 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:32828
I20240502 21:28:38.795496 11267 raft_consensus.cc:482] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:38.795668 11267 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:38.797709 11267 raft_consensus.cc:504] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:38.798390 11267 leader_election.cc:288] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 3 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721)
I20240502 21:28:38.798996 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:38.799005 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:38.799232 11190 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:38.799314 11112 raft_consensus.cc:3032] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:28:38.799440 11112 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:38.799739 11112 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:38.800155 11112 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:38.801504 11190 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 3.
I20240502 21:28:38.801970 11230 leader_election.cc:302] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:38.802280 11385 raft_consensus.cc:2781] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:28:38.802420 11112 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 3.
I20240502 21:28:38.802501 11385 raft_consensus.cc:686] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:38.802826 11385 consensus_queue.cc:237] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:38.805708 11010 catalog_manager.cc:5484] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 2 to 3, leader changed from d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65) to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 3 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:39.099730 11190 raft_consensus.cc:1261] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:39.100414 11385 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:39.105105 11112 raft_consensus.cc:1261] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:39.108419 11390 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:39.229697 11190 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:39.230418 11390 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:39.234163 11112 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:39.234910 11390 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:39.342360 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:39.344022 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:39.344251 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:40.344705 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:40.346536 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:40.346725 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:41.347232 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:41.349498 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:41.349730 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:42.350207 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:42.351922 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:42.352155 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:43.352695 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:43.355201 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:43.355471 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:44.356036 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:44.357728 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:44.357910 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:45.358425 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:45.360599 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:45.360812 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:46.361274 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:46.363118 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:46.363287 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:47.363780 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:47.365639 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:47.365826 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:48.377755 10981 auto_leader_rebalancer-test.cc:280] The leader distribution is
I20240502 21:28:48.379793 11267 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:32830
I20240502 21:28:48.380103 11267 raft_consensus.cc:595] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:48.389494 11113 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:55676
I20240502 21:28:48.389743 11113 raft_consensus.cc:595] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:48.392151 11113 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:55676
I20240502 21:28:48.392369 11113 raft_consensus.cc:595] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:48.394713 11113 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:55676
I20240502 21:28:48.394939 11113 raft_consensus.cc:595] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:48.397261 11267 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
mode: GRACEFUL
new_leader_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:32830
I20240502 21:28:48.397518 11267 raft_consensus.cc:595] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Received request to transfer leadership to TS 66ac0c62123043168247dd5c102d068e
I20240502 21:28:48.456311 11408 raft_consensus.cc:979] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.456516 11408 raft_consensus.cc:1067] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.457262 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39048
I20240502 21:28:48.457476 11190 raft_consensus.cc:482] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:48.457615 11190 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:48.459733 11190 raft_consensus.cc:504] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.460449 11190 leader_election.cc:288] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:48.460979 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:48.461014 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:48.461247 11113 raft_consensus.cc:3032] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:48.461257 11267 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:48.461395 11113 raft_consensus.cc:729] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:48.461679 11113 consensus_queue.cc:260] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.462155 11113 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:48.464570 11267 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:48.465032 11113 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:48.465133 11154 leader_election.cc:302] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:48.465492 11389 raft_consensus.cc:2781] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:48.465950 11389 raft_consensus.cc:686] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:48.466252 11389 consensus_queue.cc:237] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.469071 11010 catalog_manager.cc:5484] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 1 to 2, leader changed from d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 2 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:48.554848 11411 raft_consensus.cc:979] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.555115 11408 raft_consensus.cc:1067] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.555920 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39048
I20240502 21:28:48.556239 11190 raft_consensus.cc:482] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:48.556354 11190 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:48.558434 11190 raft_consensus.cc:504] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.559100 11190 leader_election.cc:288] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 3 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:48.559737 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:48.559813 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:48.559991 11113 raft_consensus.cc:3032] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:28:48.560026 11267 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:48.560127 11113 raft_consensus.cc:729] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:48.560436 11113 consensus_queue.cc:260] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.560818 11113 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:48.562844 11267 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 3.
I20240502 21:28:48.562847 11113 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 3.
I20240502 21:28:48.563392 11154 leader_election.cc:302] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:48.563741 11389 raft_consensus.cc:2781] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:28:48.563988 11389 raft_consensus.cc:686] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:48.564296 11389 consensus_queue.cc:237] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.567445 11010 catalog_manager.cc:5484] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 2 to 3, leader changed from d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 3 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:48.570272 11411 raft_consensus.cc:979] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.570490 11411 raft_consensus.cc:1067] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.571358 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39048
I20240502 21:28:48.571547 11190 raft_consensus.cc:482] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:48.571652 11190 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:48.573630 11190 raft_consensus.cc:504] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.574214 11190 leader_election.cc:288] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:48.574822 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:48.574827 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:48.575039 11267 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:48.575075 11113 raft_consensus.cc:3032] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:28:48.575182 11113 raft_consensus.cc:729] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:48.575471 11113 consensus_queue.cc:260] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.575835 11113 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:28:48.577280 11267 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:48.577747 11154 leader_election.cc:302] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, 66ac0c62123043168247dd5c102d068e; no voters:
I20240502 21:28:48.578095 11389 raft_consensus.cc:2781] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:28:48.578217 11113 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 2.
I20240502 21:28:48.578289 11389 raft_consensus.cc:686] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:48.578645 11389 consensus_queue.cc:237] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.581600 11010 catalog_manager.cc:5484] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 1 to 2, leader changed from d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 2 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:48.599748 11385 raft_consensus.cc:979] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.599992 11385 raft_consensus.cc:1067] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.600819 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39058
I20240502 21:28:48.601061 11190 raft_consensus.cc:482] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:48.601188 11190 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:48.603502 11190 raft_consensus.cc:504] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.604199 11190 leader_election.cc:288] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 4 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:48.604868 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:48.604897 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff778a95652e4bf1907120052e3be52a" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:48.605063 11113 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:48.605116 11267 raft_consensus.cc:3032] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 21:28:48.605229 11267 raft_consensus.cc:729] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:48.605517 11267 consensus_queue.cc:260] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.605909 11267 raft_consensus.cc:3037] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:48.607556 11113 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 4.
I20240502 21:28:48.608136 11154 leader_election.cc:302] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:48.608213 11267 raft_consensus.cc:2454] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 4.
I20240502 21:28:48.608497 11389 raft_consensus.cc:2781] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Leader election won for term 4
I20240502 21:28:48.608695 11389 raft_consensus.cc:686] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:48.608973 11389 consensus_queue.cc:237] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:48.611936 11010 catalog_manager.cc:5484] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 3 to 4, leader changed from 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 4 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:48.785046 11420 raft_consensus.cc:979] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: : Instructing follower 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.785243 11420 raft_consensus.cc:1067] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Signalling peer 66ac0c62123043168247dd5c102d068e to start an election
I20240502 21:28:48.786031 11190 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
from {username='slave'} at 127.0.0.1:39058
I20240502 21:28:48.786244 11190 raft_consensus.cc:482] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:48.786393 11190 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:48.788841 11190 raft_consensus.cc:504] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:48.789494 11190 leader_election.cc:288] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 4 election: Requested vote from peers 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337), d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721)
I20240502 21:28:48.789961 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:48.790009 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "66ac0c62123043168247dd5c102d068e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:48.790175 11113 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:48.790257 11267 raft_consensus.cc:3032] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 21:28:48.790416 11267 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:48.790726 11267 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:48.791088 11267 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:48.792534 11113 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 4.
I20240502 21:28:48.792927 11154 leader_election.cc:302] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ac0c62123043168247dd5c102d068e, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:48.793222 11389 raft_consensus.cc:2781] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Leader election won for term 4
I20240502 21:28:48.793370 11389 raft_consensus.cc:686] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Becoming Leader. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:48.793510 11267 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ac0c62123043168247dd5c102d068e in term 4.
I20240502 21:28:48.793723 11389 consensus_queue.cc:237] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:48.796609 11010 catalog_manager.cc:5484] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e reported cstate change: term changed from 3 to 4, leader changed from 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67) to 66ac0c62123043168247dd5c102d068e (127.10.185.66). New cstate: current_term: 4 leader_uuid: "66ac0c62123043168247dd5c102d068e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:48.977559 11267 raft_consensus.cc:1261] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:48.978260 11389 consensus_queue.cc:1035] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:48.986745 11113 raft_consensus.cc:1261] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:48.987895 11389 consensus_queue.cc:1035] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:49.007786 11267 raft_consensus.cc:1261] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:49.008517 11425 consensus_queue.cc:1035] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:49.012465 11113 raft_consensus.cc:1261] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:28:49.013060 11425 consensus_queue.cc:1035] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:28:49.037545 11267 raft_consensus.cc:1261] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:49.038280 11425 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:49.042270 11113 raft_consensus.cc:1261] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:49.042963 11412 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:49.163218 11113 raft_consensus.cc:1261] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:28:49.164028 11412 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:28:49.168463 11267 raft_consensus.cc:1261] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:28:49.169077 11412 consensus_queue.cc:1035] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:28:49.249723 11267 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:28:49.250411 11412 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:28:49.254940 11113 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Refusing update from remote peer 66ac0c62123043168247dd5c102d068e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:28:49.255882 11437 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:28:51.414546 10981 auto_leader_rebalancer-test.cc:295] The leader distribution is
I20240502 21:28:51.414783 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:51.417150 11190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:52416
I20240502 21:28:51.417384 11190 raft_consensus.cc:595] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Received request to transfer leadership to TS d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:51.418514 11190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:52416
I20240502 21:28:51.418745 11190 raft_consensus.cc:595] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 LEADER]: Received request to transfer leadership to TS d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:51.419754 11190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:52416
I20240502 21:28:51.420023 11190 raft_consensus.cc:595] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Received request to transfer leadership to TS 09d0afb22b4d4843859568e9b95f80d3
I20240502 21:28:51.421026 11190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:52416
I20240502 21:28:51.421283 11190 raft_consensus.cc:595] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Received request to transfer leadership to TS 09d0afb22b4d4843859568e9b95f80d3
I20240502 21:28:51.422073 11412 raft_consensus.cc:979] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: : Instructing follower d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:51.422462 11190 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644"
dest_uuid: "66ac0c62123043168247dd5c102d068e"
mode: GRACEFUL
new_leader_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:52416
I20240502 21:28:51.422644 11190 raft_consensus.cc:595] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Received request to transfer leadership to TS d811f5b5f8d84492a2990597d41cfec8
I20240502 21:28:51.422312 11412 raft_consensus.cc:1067] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Signalling peer d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:51.423076 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 5
I20240502 21:28:51.423321 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:51.423377 11113 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:36318
I20240502 21:28:51.423561 11113 raft_consensus.cc:482] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:51.423652 11113 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.425652 11113 raft_consensus.cc:504] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.426427 11113 leader_election.cc:288] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 3 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:51.426918 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:51.426925 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "046b04233e72429fa8065ddf9a5d1e2a" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:51.427217 11190 raft_consensus.cc:3032] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:28:51.427243 11267 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.427336 11190 raft_consensus.cc:729] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:51.427654 11190 consensus_queue.cc:260] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.428097 11190 raft_consensus.cc:3037] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.430497 11267 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 3.
I20240502 21:28:51.430994 11190 raft_consensus.cc:2454] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 3.
I20240502 21:28:51.431097 11078 leader_election.cc:302] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:51.431738 11441 raft_consensus.cc:2781] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:28:51.432318 11441 raft_consensus.cc:686] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 3 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:51.432636 11441 consensus_queue.cc:237] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.435551 11011 catalog_manager.cc:5484] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 2 to 3, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 3 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:51.586776 11439 raft_consensus.cc:979] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: : Instructing follower 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:51.586994 11439 raft_consensus.cc:1067] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Signalling peer 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:51.587788 11267 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:32822
I20240502 21:28:51.588073 11267 raft_consensus.cc:482] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:51.588212 11267 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.590265 11267 raft_consensus.cc:504] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.591179 11267 leader_election.cc:288] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 3 election: Requested vote from peers d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721), 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387)
I20240502 21:28:51.591727 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:51.591739 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b14e88bc4f4a3d8142745b6d9aa2b6" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:51.591991 11190 raft_consensus.cc:3032] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:28:51.592029 11113 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.592142 11190 raft_consensus.cc:729] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:51.592514 11190 consensus_queue.cc:260] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.592873 11190 raft_consensus.cc:3037] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.594467 11113 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 3.
I20240502 21:28:51.595021 11231 leader_election.cc:302] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:51.595031 11190 raft_consensus.cc:2454] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 3.
I20240502 21:28:51.595350 11420 raft_consensus.cc:2781] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:28:51.595871 11420 raft_consensus.cc:686] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:51.596211 11420 consensus_queue.cc:237] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.599236 11011 catalog_manager.cc:5484] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 2 to 3, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 3 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } }
I20240502 21:28:51.646756 11412 raft_consensus.cc:979] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: : Instructing follower d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:51.646957 11412 raft_consensus.cc:1067] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 LEADER]: Signalling peer d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:51.647747 11113 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:36318
I20240502 21:28:51.648042 11113 raft_consensus.cc:482] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:51.648166 11113 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:51.650207 11113 raft_consensus.cc:504] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.650889 11113 leader_election.cc:288] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 4 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:51.651476 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:51.651479 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d771dd765f4c5b9109da7efb384c26" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:51.651839 11190 raft_consensus.cc:3032] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 LEADER]: Stepping down as leader of term 3
I20240502 21:28:51.651866 11267 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:51.651988 11190 raft_consensus.cc:729] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:51.652346 11190 consensus_queue.cc:260] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.652771 11190 raft_consensus.cc:3037] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:28:51.655179 11267 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 4.
I20240502 21:28:51.655651 11190 raft_consensus.cc:2454] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 4.
I20240502 21:28:51.655709 11078 leader_election.cc:302] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:51.656107 11441 raft_consensus.cc:2781] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 21:28:51.656294 11441 raft_consensus.cc:686] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 4 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:51.656587 11441 consensus_queue.cc:237] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.658991 11439 raft_consensus.cc:979] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: : Instructing follower d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:51.659186 11412 raft_consensus.cc:1067] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Signalling peer d811f5b5f8d84492a2990597d41cfec8 to start an election
I20240502 21:28:51.659910 11113 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644"
dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
from {username='slave'} at 127.0.0.1:36318
I20240502 21:28:51.659790 11011 catalog_manager.cc:5484] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 3 to 4, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 4 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:51.660151 11113 raft_consensus.cc:482] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:51.660293 11113 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.662675 11113 raft_consensus.cc:504] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.663425 11113 leader_election.cc:288] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 3 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67:35337)
I20240502 21:28:51.663897 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:51.663920 11267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde1bc3bd44a429087a018d0b1ab0644" candidate_uuid: "d811f5b5f8d84492a2990597d41cfec8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
I20240502 21:28:51.664218 11267 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.664224 11190 raft_consensus.cc:3032] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:28:51.664376 11190 raft_consensus.cc:729] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:51.664693 11190 consensus_queue.cc:260] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.665068 11190 raft_consensus.cc:3037] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:28:51.666601 11267 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 3.
I20240502 21:28:51.667042 11078 leader_election.cc:302] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:51.667229 11190 raft_consensus.cc:2454] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d811f5b5f8d84492a2990597d41cfec8 in term 3.
I20240502 21:28:51.667392 11441 raft_consensus.cc:2781] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:28:51.667555 11441 raft_consensus.cc:686] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 3 LEADER]: Becoming Leader. State: Replica: d811f5b5f8d84492a2990597d41cfec8, State: Running, Role: LEADER
I20240502 21:28:51.667876 11441 consensus_queue.cc:237] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } }
I20240502 21:28:51.670644 11011 catalog_manager.cc:5484] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 reported cstate change: term changed from 2 to 3, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65). New cstate: current_term: 3 leader_uuid: "d811f5b5f8d84492a2990597d41cfec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:51.722146 11439 raft_consensus.cc:979] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: : Instructing follower 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:51.722368 11439 raft_consensus.cc:1067] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Signalling peer 09d0afb22b4d4843859568e9b95f80d3 to start an election
I20240502 21:28:51.723093 11267 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec"
dest_uuid: "09d0afb22b4d4843859568e9b95f80d3"
from {username='slave'} at 127.0.0.1:32822
I20240502 21:28:51.723305 11267 raft_consensus.cc:482] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:28:51.723415 11267 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Advancing to term 5
I20240502 21:28:51.725520 11267 raft_consensus.cc:504] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:51.726188 11267 leader_election.cc:288] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 5 election: Requested vote from peers 66ac0c62123043168247dd5c102d068e (127.10.185.66:38387), d811f5b5f8d84492a2990597d41cfec8 (127.10.185.65:41721)
I20240502 21:28:51.726734 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "66ac0c62123043168247dd5c102d068e"
I20240502 21:28:51.726773 11113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a1add9186b4b79b4219a7c71d098ec" candidate_uuid: "09d0afb22b4d4843859568e9b95f80d3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d811f5b5f8d84492a2990597d41cfec8"
I20240502 21:28:51.726999 11190 raft_consensus.cc:3032] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Stepping down as leader of term 4
I20240502 21:28:51.727008 11113 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Advancing to term 5
I20240502 21:28:51.727126 11190 raft_consensus.cc:729] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 66ac0c62123043168247dd5c102d068e, State: Running, Role: LEADER
I20240502 21:28:51.727474 11190 consensus_queue.cc:260] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:51.727823 11190 raft_consensus.cc:3037] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Advancing to term 5
I20240502 21:28:51.729583 11113 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 5.
I20240502 21:28:51.730008 11190 raft_consensus.cc:2454] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d0afb22b4d4843859568e9b95f80d3 in term 5.
I20240502 21:28:51.730103 11231 leader_election.cc:302] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d0afb22b4d4843859568e9b95f80d3, d811f5b5f8d84492a2990597d41cfec8; no voters:
I20240502 21:28:51.730473 11420 raft_consensus.cc:2781] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 21:28:51.730686 11420 raft_consensus.cc:686] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 5 LEADER]: Becoming Leader. State: Replica: 09d0afb22b4d4843859568e9b95f80d3, State: Running, Role: LEADER
I20240502 21:28:51.730998 11420 consensus_queue.cc:237] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } }
I20240502 21:28:51.733980 11011 catalog_manager.cc:5484] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 reported cstate change: term changed from 4 to 5, leader changed from 66ac0c62123043168247dd5c102d068e (127.10.185.66) to 09d0afb22b4d4843859568e9b95f80d3 (127.10.185.67). New cstate: current_term: 5 leader_uuid: "09d0afb22b4d4843859568e9b95f80d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 } health_report { overall_health: UNKNOWN } } }
I20240502 21:28:51.983436 11190 raft_consensus.cc:1261] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:51.984126 11441 consensus_queue.cc:1035] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:51.989727 11267 raft_consensus.cc:1261] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:51.990593 11441 consensus_queue.cc:1035] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:52.084914 11190 raft_consensus.cc:1261] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:52.085701 11456 consensus_queue.cc:1035] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:52.090023 11267 raft_consensus.cc:1261] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:52.091948 11442 consensus_queue.cc:1035] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:52.100867 11113 raft_consensus.cc:1261] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:52.101583 11420 consensus_queue.cc:1035] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:52.106213 11190 raft_consensus.cc:1261] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:28:52.106915 11420 consensus_queue.cc:1035] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:28:52.147090 11267 raft_consensus.cc:1261] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:28:52.147785 11460 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d0afb22b4d4843859568e9b95f80d3" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:28:52.151947 11190 raft_consensus.cc:1261] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Refusing update from remote peer d811f5b5f8d84492a2990597d41cfec8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:28:52.152572 11456 consensus_queue.cc:1035] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:28:52.281517 11190 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 5 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240502 21:28:52.282143 11463 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ac0c62123043168247dd5c102d068e" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 21:28:52.286338 11113 raft_consensus.cc:1261] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 5 FOLLOWER]: Refusing update from remote peer 09d0afb22b4d4843859568e9b95f80d3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240502 21:28:52.287043 11448 consensus_queue.cc:1035] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811f5b5f8d84492a2990597d41cfec8" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 21:28:52.423916 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:52.425662 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:52.425905 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:53.426328 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:53.427943 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:53.428186 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:54.428665 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:54.430718 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:54.430946 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:55.431455 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:55.433248 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:55.433465 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:56.433926 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:56.435873 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:56.436097 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:57.436532 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:57.438117 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:57.438285 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:58.438838 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:58.441289 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:58.441574 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:28:59.442092 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:28:59.443746 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:28:59.443922 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:00.444438 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:00.446388 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:00.446620 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:01.458886 10981 auto_leader_rebalancer-test.cc:306] The leader distribution is
I20240502 21:29:01.459223 10981 tablet_server.cc:178] TabletServer@127.10.185.65:0 shutting down...
I20240502 21:29:01.473346 10981 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240502 21:29:01.473717 10981 tablet_replica.cc:330] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.474156 10981 raft_consensus.cc:2229] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.474468 10981 raft_consensus.cc:2258] T ff778a95652e4bf1907120052e3be52a P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.476284 10981 tablet_replica.cc:330] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.476513 10981 raft_consensus.cc:2229] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.476707 10981 raft_consensus.cc:2258] T 84a1add9186b4b79b4219a7c71d098ec P d811f5b5f8d84492a2990597d41cfec8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.477304 10981 tablet_replica.cc:330] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.477439 10981 raft_consensus.cc:2229] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 3 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.477783 10981 raft_consensus.cc:2258] T cde1bc3bd44a429087a018d0b1ab0644 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.478313 10981 tablet_replica.cc:330] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.478483 10981 raft_consensus.cc:2229] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.478638 10981 raft_consensus.cc:2258] T f93f4a1cf91841438cd87199c63b07b5 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.479146 10981 tablet_replica.cc:330] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.479274 10981 raft_consensus.cc:2229] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.479456 10981 raft_consensus.cc:2258] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.479884 10981 tablet_replica.cc:330] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.480110 10981 raft_consensus.cc:2229] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.480257 10981 raft_consensus.cc:2258] T 5c7e1651bcb84389ad070db5b722d0ab P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.480700 10981 tablet_replica.cc:330] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.480842 10981 raft_consensus.cc:2229] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 3 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.481139 10981 raft_consensus.cc:2258] T 046b04233e72429fa8065ddf9a5d1e2a P d811f5b5f8d84492a2990597d41cfec8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.481570 10981 tablet_replica.cc:330] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.481722 10981 raft_consensus.cc:2229] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.481869 10981 raft_consensus.cc:2258] T 51b42b2522314532b39caf9705afe670 P d811f5b5f8d84492a2990597d41cfec8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.482304 10981 tablet_replica.cc:330] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8: stopping tablet replica
I20240502 21:29:01.482430 10981 raft_consensus.cc:2229] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 4 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.482709 10981 raft_consensus.cc:2258] T 06d771dd765f4c5b9109da7efb384c26 P d811f5b5f8d84492a2990597d41cfec8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.500334 10981 tablet_server.cc:194] TabletServer@127.10.185.65:0 shutdown complete.
I20240502 21:29:01.507629 10981 tablet_server.cc:178] TabletServer@127.10.185.66:0 shutting down...
I20240502 21:29:01.517292 10981 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240502 21:29:01.517607 10981 tablet_replica.cc:330] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.517787 10981 raft_consensus.cc:2229] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.518113 10981 raft_consensus.cc:2258] T 5c7e1651bcb84389ad070db5b722d0ab P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.518713 10981 tablet_replica.cc:330] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.518885 10981 raft_consensus.cc:2229] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 2 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.519145 10981 raft_consensus.cc:2258] T f93f4a1cf91841438cd87199c63b07b5 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.519614 10981 tablet_replica.cc:330] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.519733 10981 raft_consensus.cc:2229] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.519883 10981 raft_consensus.cc:2258] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.520371 10981 tablet_replica.cc:330] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.520504 10981 raft_consensus.cc:2229] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 5 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.520625 10981 raft_consensus.cc:2258] T 84a1add9186b4b79b4219a7c71d098ec P 66ac0c62123043168247dd5c102d068e [term 5 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.521023 10981 tablet_replica.cc:330] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.521142 10981 raft_consensus.cc:2229] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.521270 10981 raft_consensus.cc:2258] T cde1bc3bd44a429087a018d0b1ab0644 P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.521633 10981 tablet_replica.cc:330] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.521762 10981 raft_consensus.cc:2229] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.521891 10981 raft_consensus.cc:2258] T 046b04233e72429fa8065ddf9a5d1e2a P 66ac0c62123043168247dd5c102d068e [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.522266 10981 tablet_replica.cc:330] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.522377 10981 raft_consensus.cc:2229] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 4 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.522647 10981 raft_consensus.cc:2258] T ff778a95652e4bf1907120052e3be52a P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.523044 10981 tablet_replica.cc:330] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.523169 10981 raft_consensus.cc:2229] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.523276 10981 raft_consensus.cc:2258] T 06d771dd765f4c5b9109da7efb384c26 P 66ac0c62123043168247dd5c102d068e [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.523638 10981 tablet_replica.cc:330] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e: stopping tablet replica
I20240502 21:29:01.523742 10981 raft_consensus.cc:2229] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.523881 10981 raft_consensus.cc:2258] T 51b42b2522314532b39caf9705afe670 P 66ac0c62123043168247dd5c102d068e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.540092 10981 tablet_server.cc:194] TabletServer@127.10.185.66:0 shutdown complete.
I20240502 21:29:01.544682 10981 tablet_server.cc:178] TabletServer@127.10.185.67:0 shutting down...
I20240502 21:29:01.554826 10981 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240502 21:29:01.555227 10981 tablet_replica.cc:330] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.555464 10981 raft_consensus.cc:2229] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.555702 10981 raft_consensus.cc:2258] T f93f4a1cf91841438cd87199c63b07b5 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.556393 10981 tablet_replica.cc:330] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.556600 10981 raft_consensus.cc:2229] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.556780 10981 raft_consensus.cc:2258] T cde1bc3bd44a429087a018d0b1ab0644 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.557413 10981 tablet_replica.cc:330] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.557602 10981 raft_consensus.cc:2229] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 5 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.557950 10981 raft_consensus.cc:2258] T 84a1add9186b4b79b4219a7c71d098ec P 09d0afb22b4d4843859568e9b95f80d3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.558475 10981 tablet_replica.cc:330] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.558666 10981 raft_consensus.cc:2229] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.558964 10981 raft_consensus.cc:2258] T b3b14e88bc4f4a3d8142745b6d9aa2b6 P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.559482 10981 tablet_replica.cc:330] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.559643 10981 raft_consensus.cc:2229] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.559827 10981 raft_consensus.cc:2258] T 5c7e1651bcb84389ad070db5b722d0ab P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.560338 10981 tablet_replica.cc:330] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.560508 10981 raft_consensus.cc:2229] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.560679 10981 raft_consensus.cc:2258] T 046b04233e72429fa8065ddf9a5d1e2a P 09d0afb22b4d4843859568e9b95f80d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.561157 10981 tablet_replica.cc:330] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.561282 10981 raft_consensus.cc:2229] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.561585 10981 raft_consensus.cc:2258] T 51b42b2522314532b39caf9705afe670 P 09d0afb22b4d4843859568e9b95f80d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.562040 10981 tablet_replica.cc:330] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.562177 10981 raft_consensus.cc:2229] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.562307 10981 raft_consensus.cc:2258] T 06d771dd765f4c5b9109da7efb384c26 P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.562706 10981 tablet_replica.cc:330] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3: stopping tablet replica
I20240502 21:29:01.562815 10981 raft_consensus.cc:2229] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:01.562956 10981 raft_consensus.cc:2258] T ff778a95652e4bf1907120052e3be52a P 09d0afb22b4d4843859568e9b95f80d3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.579924 10981 tablet_server.cc:194] TabletServer@127.10.185.67:0 shutdown complete.
I20240502 21:29:01.584836 10981 master.cc:537] Master@127.10.185.126:33961 shutting down...
I20240502 21:29:01.594148 10981 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 1 LEADER]: Raft consensus shutting down.
I20240502 21:29:01.594470 10981 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:01.594609 10981 tablet_replica.cc:330] T 00000000000000000000000000000000 P e91dc03c15894f9e8375117ebda800de: stopping tablet replica
I20240502 21:29:01.608830 10981 master.cc:552] Master@127.10.185.126:33961 shutdown complete.
09d0afb22b4d4843859568e9b95f80d3 1
66ac0c62123043168247dd5c102d068e 4
d811f5b5f8d84492a2990597d41cfec8 4
09d0afb22b4d4843859568e9b95f80d3 3
66ac0c62123043168247dd5c102d068e 3
d811f5b5f8d84492a2990597d41cfec8 3
09d0afb22b4d4843859568e9b95f80d3 1
66ac0c62123043168247dd5c102d068e 8
d811f5b5f8d84492a2990597d41cfec8 0
09d0afb22b4d4843859568e9b95f80d3 3
66ac0c62123043168247dd5c102d068e 3
d811f5b5f8d84492a2990597d41cfec8 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (27516 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20240502 21:29:01.629184 10981 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.185.126:33339
I20240502 21:29:01.629840 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:01.633384 11477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.634162 11478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.635041 11480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:01.635705 10981 server_base.cc:1034] running on GCE node
I20240502 21:29:01.636201 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:01.636286 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:01.636373 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685341636334 us; error 0 us; skew 500 ppm
I20240502 21:29:01.636569 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:01.643929 10981 webserver.cc:460] Webserver started at http://127.10.185.126:39215/ using document root <none> and password file <none>
I20240502 21:29:01.644260 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:01.644371 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:01.644531 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:01.645318 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/master-0-root/instance:
uuid: "3f69d2c980a34635b9467d373bbc0f85"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.648319 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20240502 21:29:01.650543 11485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.651017 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20240502 21:29:01.651161 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/master-0-root
uuid: "3f69d2c980a34635b9467d373bbc0f85"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.651338 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:01.669359 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:01.670104 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:01.688058 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.126:33339
I20240502 21:29:01.688076 11536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.126:33339 every 8 connection(s)
I20240502 21:29:01.689683 11537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:01.694615 11537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85: Bootstrap starting.
I20240502 21:29:01.696578 11537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:01.698941 11537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85: No bootstrap required, opened a new log
I20240502 21:29:01.699820 11537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f69d2c980a34635b9467d373bbc0f85" member_type: VOTER }
I20240502 21:29:01.700067 11537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:01.700208 11537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f69d2c980a34635b9467d373bbc0f85, State: Initialized, Role: FOLLOWER
I20240502 21:29:01.700517 11537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f69d2c980a34635b9467d373bbc0f85" member_type: VOTER }
I20240502 21:29:01.700702 11537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 21:29:01.700791 11537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 21:29:01.700934 11537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:01.703058 11537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f69d2c980a34635b9467d373bbc0f85" member_type: VOTER }
I20240502 21:29:01.703339 11537 leader_election.cc:302] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f69d2c980a34635b9467d373bbc0f85; no voters:
I20240502 21:29:01.704125 11537 leader_election.cc:288] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 21:29:01.704380 11540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:01.704941 11540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 1 LEADER]: Becoming Leader. State: Replica: 3f69d2c980a34635b9467d373bbc0f85, State: Running, Role: LEADER
I20240502 21:29:01.705286 11537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 21:29:01.705328 11540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f69d2c980a34635b9467d373bbc0f85" member_type: VOTER }
I20240502 21:29:01.706609 11542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f69d2c980a34635b9467d373bbc0f85. Latest consensus state: current_term: 1 leader_uuid: "3f69d2c980a34635b9467d373bbc0f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f69d2c980a34635b9467d373bbc0f85" member_type: VOTER } }
I20240502 21:29:01.706563 11541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f69d2c980a34635b9467d373bbc0f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f69d2c980a34635b9467d373bbc0f85" member_type: VOTER } }
I20240502 21:29:01.706825 11542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [sys.catalog]: This master's current role is: LEADER
I20240502 21:29:01.707011 11541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [sys.catalog]: This master's current role is: LEADER
I20240502 21:29:01.711484 10981 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20240502 21:29:01.713007 11556 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 21:29:01.713126 11556 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 21:29:01.713274 11545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 21:29:01.714896 11545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 21:29:01.718396 11545 catalog_manager.cc:1347] Generated new cluster ID: c6364789208e430d9d95ee56263566dd
I20240502 21:29:01.718542 11545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 21:29:01.726239 11545 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 21:29:01.727157 11545 catalog_manager.cc:1504] Loading token signing keys...
I20240502 21:29:01.743801 11545 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85: Generated new TSK 0
I20240502 21:29:01.744131 11545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 21:29:01.777025 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:01.780776 11558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.781615 11559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:01.782346 10981 server_base.cc:1034] running on GCE node
W20240502 21:29:01.782877 11561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:01.783313 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:01.783387 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:01.783427 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685341783412 us; error 0 us; skew 500 ppm
I20240502 21:29:01.783600 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:01.785115 10981 webserver.cc:460] Webserver started at http://127.10.185.65:33593/ using document root <none> and password file <none>
I20240502 21:29:01.785377 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:01.785454 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:01.785591 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:01.786311 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-0-root/instance:
uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.789192 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 21:29:01.791069 11566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.791489 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20240502 21:29:01.791639 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-0-root
uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.791776 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:01.803857 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:01.804632 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:01.805271 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:01.806257 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:01.806357 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.806483 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:01.806558 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.824347 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.65:45333
I20240502 21:29:01.824401 11630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.65:45333 every 8 connection(s)
I20240502 21:29:01.826311 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:01.831116 11635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.832026 11636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.832935 11638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:01.833622 10981 server_base.cc:1034] running on GCE node
I20240502 21:29:01.833951 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:01.834023 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:01.834103 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685341834047 us; error 0 us; skew 500 ppm
I20240502 21:29:01.834283 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:01.835292 11631 heartbeater.cc:347] Connected to a master server at 127.10.185.126:33339
I20240502 21:29:01.835450 11631 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:01.835808 10981 webserver.cc:460] Webserver started at http://127.10.185.66:46221/ using document root <none> and password file <none>
I20240502 21:29:01.835955 11631 heartbeater.cc:511] Master 127.10.185.126:33339 requested a full tablet report, sending...
I20240502 21:29:01.836123 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:01.836242 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:01.836395 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:01.837086 11502 ts_manager.cc:194] Registered new tserver with Master: fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333)
I20240502 21:29:01.837167 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-1-root/instance:
uuid: "47a8645e1fc349299fd909d30cf01dd1"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.838392 11502 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60504
I20240502 21:29:01.840049 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 21:29:01.841835 11643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.842422 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240502 21:29:01.842566 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-1-root
uuid: "47a8645e1fc349299fd909d30cf01dd1"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.842725 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:01.862470 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:01.863173 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:01.863798 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:01.864781 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:01.864859 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.865002 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:01.865064 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.881832 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.66:45427
I20240502 21:29:01.881860 11705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.66:45427 every 8 connection(s)
I20240502 21:29:01.883746 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:01.888235 11709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.888648 11710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:01.889109 11712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:01.892205 11706 heartbeater.cc:347] Connected to a master server at 127.10.185.126:33339
I20240502 21:29:01.892474 11706 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:01.893361 11706 heartbeater.cc:511] Master 127.10.185.126:33339 requested a full tablet report, sending...
I20240502 21:29:01.894538 11502 ts_manager.cc:194] Registered new tserver with Master: 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:01.895319 10981 server_base.cc:1034] running on GCE node
I20240502 21:29:01.895648 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20240502 21:29:01.895705 11502 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60506
W20240502 21:29:01.895768 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:01.895828 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685341895804 us; error 0 us; skew 500 ppm
I20240502 21:29:01.896075 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:01.897572 10981 webserver.cc:460] Webserver started at http://127.10.185.67:34217/ using document root <none> and password file <none>
I20240502 21:29:01.897830 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:01.897912 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:01.898077 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:01.898743 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-2-root/instance:
uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.901538 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:01.903355 11717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.903769 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240502 21:29:01.903932 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-2-root
uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
format_stamp: "Formatted at 2024-05-02 21:29:01 on dist-test-slave-ccs3"
I20240502 21:29:01.904124 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714685314041113-10981-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:01.912544 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:01.913184 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:01.913856 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:01.914847 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:01.914952 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.915064 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:01.915143 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:01.934907 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.67:46519
I20240502 21:29:01.934945 11781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.67:46519 every 8 connection(s)
I20240502 21:29:01.942801 11782 heartbeater.cc:347] Connected to a master server at 127.10.185.126:33339
I20240502 21:29:01.942982 11782 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:01.943421 11782 heartbeater.cc:511] Master 127.10.185.126:33339 requested a full tablet report, sending...
I20240502 21:29:01.944725 11502 ts_manager.cc:194] Registered new tserver with Master: 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:01.945413 10981 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.009379005s
I20240502 21:29:01.945711 11502 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60514
I20240502 21:29:01.946810 10981 test_util.cc:268] Using random seed: 1253397404
I20240502 21:29:01.961411 11502 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60522:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240502 21:29:01.962414 11502 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240502 21:29:02.013466 11746 tablet_service.cc:1467] Processing CreateTablet for tablet 61d83bba66524f9288af49392ab4ed96 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240502 21:29:02.014550 11746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d83bba66524f9288af49392ab4ed96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.014926 11670 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4317ce6a1b451a84ae2ced0fc8ae6d (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240502 21:29:02.015486 11745 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4317ce6a1b451a84ae2ced0fc8ae6d (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240502 21:29:02.016290 11744 tablet_service.cc:1467] Processing CreateTablet for tablet 272a4fe221da43439bc128eb339ef434 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240502 21:29:02.017233 11670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4317ce6a1b451a84ae2ced0fc8ae6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.016975 11745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4317ce6a1b451a84ae2ced0fc8ae6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.016973 11668 tablet_service.cc:1467] Processing CreateTablet for tablet 0059e9d031fa4b5c9ad9414f80a61529 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240502 21:29:02.017868 11744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 272a4fe221da43439bc128eb339ef434. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.019065 11742 tablet_service.cc:1467] Processing CreateTablet for tablet 0defef714aa44459a0a1a1a6468dfe0f (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240502 21:29:02.021063 11668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0059e9d031fa4b5c9ad9414f80a61529. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.022114 11742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0defef714aa44459a0a1a1a6468dfe0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.022114 11666 tablet_service.cc:1467] Processing CreateTablet for tablet df366396c6044cf4b642bcc2da2046b3 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240502 21:29:02.022938 11739 tablet_service.cc:1467] Processing CreateTablet for tablet 8839111f2ddd4ae28be77b16966e659f (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240502 21:29:02.023422 11738 tablet_service.cc:1467] Processing CreateTablet for tablet 085e2f4b603744e1bf1c1394a54dc9bc (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240502 21:29:02.024083 11740 tablet_service.cc:1467] Processing CreateTablet for tablet 9a990045515e4479b625327a285c6af3 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240502 21:29:02.024284 11666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df366396c6044cf4b642bcc2da2046b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.025106 11667 tablet_service.cc:1467] Processing CreateTablet for tablet 0defef714aa44459a0a1a1a6468dfe0f (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240502 21:29:02.025316 11739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8839111f2ddd4ae28be77b16966e659f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.026574 11665 tablet_service.cc:1467] Processing CreateTablet for tablet 9a990045515e4479b625327a285c6af3 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240502 21:29:02.028154 11741 tablet_service.cc:1467] Processing CreateTablet for tablet df366396c6044cf4b642bcc2da2046b3 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240502 21:29:02.028199 11671 tablet_service.cc:1467] Processing CreateTablet for tablet 61d83bba66524f9288af49392ab4ed96 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240502 21:29:02.030633 11738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 085e2f4b603744e1bf1c1394a54dc9bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.029592 11669 tablet_service.cc:1467] Processing CreateTablet for tablet 272a4fe221da43439bc128eb339ef434 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240502 21:29:02.031703 11667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0defef714aa44459a0a1a1a6468dfe0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.032826 11737 tablet_service.cc:1467] Processing CreateTablet for tablet 9da63340e627438f895d2a7edd31e5c7 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240502 21:29:02.034961 11740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a990045515e4479b625327a285c6af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.035080 11665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a990045515e4479b625327a285c6af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.032776 11743 tablet_service.cc:1467] Processing CreateTablet for tablet 0059e9d031fa4b5c9ad9414f80a61529 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240502 21:29:02.034863 11662 tablet_service.cc:1467] Processing CreateTablet for tablet 9da63340e627438f895d2a7edd31e5c7 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240502 21:29:02.038163 11669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 272a4fe221da43439bc128eb339ef434. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.036672 11595 tablet_service.cc:1467] Processing CreateTablet for tablet 61d83bba66524f9288af49392ab4ed96 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240502 21:29:02.042763 11741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df366396c6044cf4b642bcc2da2046b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.045218 11671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d83bba66524f9288af49392ab4ed96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.038570 11664 tablet_service.cc:1467] Processing CreateTablet for tablet 8839111f2ddd4ae28be77b16966e659f (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240502 21:29:02.042541 11663 tablet_service.cc:1467] Processing CreateTablet for tablet 085e2f4b603744e1bf1c1394a54dc9bc (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240502 21:29:02.047500 11595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d83bba66524f9288af49392ab4ed96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.043181 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 9a990045515e4479b625327a285c6af3 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240502 21:29:02.044102 11593 tablet_service.cc:1467] Processing CreateTablet for tablet 272a4fe221da43439bc128eb339ef434 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240502 21:29:02.047792 11737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da63340e627438f895d2a7edd31e5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.049877 11662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da63340e627438f895d2a7edd31e5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.045751 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 0defef714aa44459a0a1a1a6468dfe0f (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240502 21:29:02.046775 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 9da63340e627438f895d2a7edd31e5c7 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240502 21:29:02.051373 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 8839111f2ddd4ae28be77b16966e659f (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240502 21:29:02.052881 11743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0059e9d031fa4b5c9ad9414f80a61529. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.054667 11664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8839111f2ddd4ae28be77b16966e659f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.055415 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da63340e627438f895d2a7edd31e5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.049738 11594 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4317ce6a1b451a84ae2ced0fc8ae6d (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240502 21:29:02.049821 11592 tablet_service.cc:1467] Processing CreateTablet for tablet 0059e9d031fa4b5c9ad9414f80a61529 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240502 21:29:02.057680 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a990045515e4479b625327a285c6af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.057487 11663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 085e2f4b603744e1bf1c1394a54dc9bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.054644 11590 tablet_service.cc:1467] Processing CreateTablet for tablet df366396c6044cf4b642bcc2da2046b3 (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240502 21:29:02.057580 11587 tablet_service.cc:1467] Processing CreateTablet for tablet 085e2f4b603744e1bf1c1394a54dc9bc (DEFAULT_TABLE table=test-workload [id=9e28ce04948e40529cc5886457c1a5f6]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240502 21:29:02.063295 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0defef714aa44459a0a1a1a6468dfe0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.063608 11594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4317ce6a1b451a84ae2ced0fc8ae6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.063855 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8839111f2ddd4ae28be77b16966e659f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.067132 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 085e2f4b603744e1bf1c1394a54dc9bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.068554 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df366396c6044cf4b642bcc2da2046b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.071691 11593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 272a4fe221da43439bc128eb339ef434. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.073496 11592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0059e9d031fa4b5c9ad9414f80a61529. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:02.154795 11797 tablet_bootstrap.cc:492] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.166252 11798 tablet_bootstrap.cc:492] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.177529 11799 tablet_bootstrap.cc:492] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.183017 11797 tablet_bootstrap.cc:654] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.187920 11798 tablet_bootstrap.cc:654] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.188899 11799 tablet_bootstrap.cc:654] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.190479 11798 tablet_bootstrap.cc:492] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.190651 11798 ts_tablet_manager.cc:1367] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.005s
I20240502 21:29:02.190725 11797 tablet_bootstrap.cc:492] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.190862 11797 ts_tablet_manager.cc:1367] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.002s
I20240502 21:29:02.191437 11798 raft_consensus.cc:348] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.191476 11797 raft_consensus.cc:348] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.191654 11798 raft_consensus.cc:374] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.191720 11797 raft_consensus.cc:374] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.191764 11798 raft_consensus.cc:729] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.191839 11797 raft_consensus.cc:729] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.192179 11798 consensus_queue.cc:260] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.192443 11799 tablet_bootstrap.cc:492] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.192184 11797 consensus_queue.cc:260] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.192634 11799 ts_tablet_manager.cc:1367] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20240502 21:29:02.193950 11799 raft_consensus.cc:348] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.194242 11799 raft_consensus.cc:374] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.194375 11799 raft_consensus.cc:729] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.194777 11799 consensus_queue.cc:260] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.196003 11799 ts_tablet_manager.cc:1398] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 21:29:02.196048 11631 heartbeater.cc:503] Master 127.10.185.126:33339 was elected leader, sending a full tablet report...
I20240502 21:29:02.196449 11799 tablet_bootstrap.cc:492] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.202461 11798 ts_tablet_manager.cc:1398] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20240502 21:29:02.202580 11782 heartbeater.cc:503] Master 127.10.185.126:33339 was elected leader, sending a full tablet report...
I20240502 21:29:02.202984 11798 tablet_bootstrap.cc:492] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.204836 11799 tablet_bootstrap.cc:654] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.208452 11798 tablet_bootstrap.cc:654] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.208909 11706 heartbeater.cc:503] Master 127.10.185.126:33339 was elected leader, sending a full tablet report...
I20240502 21:29:02.210464 11797 ts_tablet_manager.cc:1398] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.019s user 0.014s sys 0.005s
I20240502 21:29:02.210924 11797 tablet_bootstrap.cc:492] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.211337 11799 tablet_bootstrap.cc:492] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.211620 11799 ts_tablet_manager.cc:1367] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.004s
I20240502 21:29:02.213150 11799 raft_consensus.cc:348] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.213485 11799 raft_consensus.cc:374] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.213583 11799 raft_consensus.cc:729] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.213848 11799 consensus_queue.cc:260] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.214522 11799 ts_tablet_manager.cc:1398] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20240502 21:29:02.214962 11799 tablet_bootstrap.cc:492] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.216718 11798 tablet_bootstrap.cc:492] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.216921 11798 ts_tablet_manager.cc:1367] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20240502 21:29:02.217901 11798 raft_consensus.cc:348] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.218189 11798 raft_consensus.cc:374] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.218295 11798 raft_consensus.cc:729] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.218600 11797 tablet_bootstrap.cc:654] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.218644 11798 consensus_queue.cc:260] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.219306 11798 ts_tablet_manager.cc:1398] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:02.219650 11798 tablet_bootstrap.cc:492] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.220918 11797 tablet_bootstrap.cc:492] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.221112 11797 ts_tablet_manager.cc:1367] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20240502 21:29:02.221411 11798 tablet_bootstrap.cc:654] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.222067 11797 raft_consensus.cc:348] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.222319 11797 raft_consensus.cc:374] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.216958 11799 tablet_bootstrap.cc:654] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.222427 11797 raft_consensus.cc:729] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.222800 11797 consensus_queue.cc:260] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.223479 11797 ts_tablet_manager.cc:1398] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 21:29:02.223935 11797 tablet_bootstrap.cc:492] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.224830 11799 tablet_bootstrap.cc:492] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.225033 11799 ts_tablet_manager.cc:1367] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20240502 21:29:02.225960 11799 raft_consensus.cc:348] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.226202 11799 raft_consensus.cc:374] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.226315 11799 raft_consensus.cc:729] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.226550 11797 tablet_bootstrap.cc:654] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.226641 11799 consensus_queue.cc:260] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.227903 11799 ts_tablet_manager.cc:1398] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240502 21:29:02.228272 11799 tablet_bootstrap.cc:492] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.229275 11798 tablet_bootstrap.cc:492] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.229477 11798 ts_tablet_manager.cc:1367] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20240502 21:29:02.230139 11799 tablet_bootstrap.cc:654] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.230703 11798 raft_consensus.cc:348] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.231021 11798 raft_consensus.cc:374] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.231151 11798 raft_consensus.cc:729] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.231570 11798 consensus_queue.cc:260] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.232263 11798 ts_tablet_manager.cc:1398] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240502 21:29:02.232599 11798 tablet_bootstrap.cc:492] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.232729 11797 tablet_bootstrap.cc:492] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.232859 11797 ts_tablet_manager.cc:1367] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20240502 21:29:02.233748 11797 raft_consensus.cc:348] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.233969 11797 raft_consensus.cc:374] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.234074 11797 raft_consensus.cc:729] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.234525 11798 tablet_bootstrap.cc:654] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.234373 11797 consensus_queue.cc:260] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.235911 11797 ts_tablet_manager.cc:1398] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240502 21:29:02.236318 11797 tablet_bootstrap.cc:492] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.237937 11804 raft_consensus.cc:482] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.238109 11804 raft_consensus.cc:504] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.239284 11804 leader_election.cc:288] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.249459 11797 tablet_bootstrap.cc:654] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.249850 11756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
W20240502 21:29:02.250797 11647 leader_election.cc:341] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:02.251586 11798 tablet_bootstrap.cc:492] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.251827 11798 ts_tablet_manager.cc:1367] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.005s
I20240502 21:29:02.253060 11798 raft_consensus.cc:348] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.253386 11798 raft_consensus.cc:374] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.253494 11798 raft_consensus.cc:729] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.253942 11798 consensus_queue.cc:260] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.256260 11798 ts_tablet_manager.cc:1398] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:02.256705 11798 tablet_bootstrap.cc:492] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.257707 11799 tablet_bootstrap.cc:492] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.257913 11799 ts_tablet_manager.cc:1367] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.017s
I20240502 21:29:02.258626 11798 tablet_bootstrap.cc:654] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.258805 11799 raft_consensus.cc:348] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.259090 11799 raft_consensus.cc:374] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.259202 11799 raft_consensus.cc:729] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.259532 11799 consensus_queue.cc:260] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.260829 11799 ts_tablet_manager.cc:1398] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:02.261170 11799 tablet_bootstrap.cc:492] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.261281 11798 tablet_bootstrap.cc:492] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.261454 11798 ts_tablet_manager.cc:1367] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 21:29:02.262197 11798 raft_consensus.cc:348] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.262445 11798 raft_consensus.cc:374] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.262565 11798 raft_consensus.cc:729] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.262831 11798 consensus_queue.cc:260] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.262991 11799 tablet_bootstrap.cc:654] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.263811 11798 ts_tablet_manager.cc:1398] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 21:29:02.264179 11798 tablet_bootstrap.cc:492] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.264305 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.264986 11797 tablet_bootstrap.cc:492] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.265178 11797 ts_tablet_manager.cc:1367] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.000s
W20240502 21:29:02.265797 11644 leader_election.cc:341] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:02.266000 11644 leader_election.cc:302] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1; no voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.266134 11798 tablet_bootstrap.cc:654] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.266005 11797 raft_consensus.cc:348] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.266523 11797 raft_consensus.cc:374] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.266628 11797 raft_consensus.cc:729] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.266680 11804 raft_consensus.cc:2726] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:02.266930 11797 consensus_queue.cc:260] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.267668 11797 ts_tablet_manager.cc:1398] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 21:29:02.273651 11797 tablet_bootstrap.cc:492] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.274595 11798 tablet_bootstrap.cc:492] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.274825 11798 ts_tablet_manager.cc:1367] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20240502 21:29:02.275202 11799 tablet_bootstrap.cc:492] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.275348 11799 ts_tablet_manager.cc:1367] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20240502 21:29:02.275842 11798 raft_consensus.cc:348] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.276089 11799 raft_consensus.cc:348] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.276175 11798 raft_consensus.cc:374] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.276271 11799 raft_consensus.cc:374] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.276288 11798 raft_consensus.cc:729] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.276351 11799 raft_consensus.cc:729] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.276476 11797 tablet_bootstrap.cc:654] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.276666 11798 consensus_queue.cc:260] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.276669 11799 consensus_queue.cc:260] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.277407 11799 ts_tablet_manager.cc:1398] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:02.277767 11799 tablet_bootstrap.cc:492] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.279434 11798 ts_tablet_manager.cc:1398] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:02.279820 11798 tablet_bootstrap.cc:492] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.279939 11799 tablet_bootstrap.cc:654] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.282084 11798 tablet_bootstrap.cc:654] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.283409 11797 tablet_bootstrap.cc:492] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.283660 11797 ts_tablet_manager.cc:1367] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.003s
I20240502 21:29:02.284126 11799 tablet_bootstrap.cc:492] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.284322 11799 ts_tablet_manager.cc:1367] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20240502 21:29:02.284554 11797 raft_consensus.cc:348] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.284752 11797 raft_consensus.cc:374] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.284888 11798 tablet_bootstrap.cc:492] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.284952 11797 raft_consensus.cc:729] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.285104 11798 ts_tablet_manager.cc:1367] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20240502 21:29:02.285194 11799 raft_consensus.cc:348] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.285295 11797 consensus_queue.cc:260] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.285528 11799 raft_consensus.cc:374] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.285637 11799 raft_consensus.cc:729] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.285979 11797 ts_tablet_manager.cc:1398] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20240502 21:29:02.286015 11799 consensus_queue.cc:260] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.286295 11797 tablet_bootstrap.cc:492] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.286805 11798 raft_consensus.cc:348] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.287052 11798 raft_consensus.cc:374] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.287206 11798 raft_consensus.cc:729] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.288028 11799 ts_tablet_manager.cc:1398] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:02.288364 11797 tablet_bootstrap.cc:654] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.288431 11799 tablet_bootstrap.cc:492] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.287508 11798 consensus_queue.cc:260] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.289067 11798 ts_tablet_manager.cc:1398] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 21:29:02.289499 11798 tablet_bootstrap.cc:492] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.290270 11799 tablet_bootstrap.cc:654] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.291587 11798 tablet_bootstrap.cc:654] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.294377 11798 tablet_bootstrap.cc:492] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.294731 11798 ts_tablet_manager.cc:1367] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20240502 21:29:02.296020 11798 raft_consensus.cc:348] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.296480 11798 raft_consensus.cc:374] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.296619 11798 raft_consensus.cc:729] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.297025 11798 consensus_queue.cc:260] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.297488 11797 tablet_bootstrap.cc:492] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.297685 11797 ts_tablet_manager.cc:1367] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.004s
I20240502 21:29:02.297721 11798 ts_tablet_manager.cc:1398] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240502 21:29:02.298100 11799 tablet_bootstrap.cc:492] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.298269 11799 ts_tablet_manager.cc:1367] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20240502 21:29:02.298329 11798 tablet_bootstrap.cc:492] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.298586 11797 raft_consensus.cc:348] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.298827 11797 raft_consensus.cc:374] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.298949 11797 raft_consensus.cc:729] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.299154 11799 raft_consensus.cc:348] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.299408 11799 raft_consensus.cc:374] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.299341 11797 consensus_queue.cc:260] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.299515 11799 raft_consensus.cc:729] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.300036 11799 consensus_queue.cc:260] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.300601 11798 tablet_bootstrap.cc:654] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.301693 11803 raft_consensus.cc:482] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.301887 11803 raft_consensus.cc:504] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.302314 11799 ts_tablet_manager.cc:1398] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:02.302641 11797 ts_tablet_manager.cc:1398] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20240502 21:29:02.302662 11799 tablet_bootstrap.cc:492] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.303033 11803 leader_election.cc:288] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.303864 11797 tablet_bootstrap.cc:492] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.306267 11797 tablet_bootstrap.cc:654] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.308203 11799 tablet_bootstrap.cc:654] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.314957 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.315112 11797 tablet_bootstrap.cc:492] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.315330 11797 ts_tablet_manager.cc:1367] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20240502 21:29:02.315295 11605 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.315909 11718 leader_election.cc:302] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.316332 11803 raft_consensus.cc:2781] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.316495 11803 raft_consensus.cc:482] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.316637 11803 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.317173 11799 tablet_bootstrap.cc:492] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.317384 11799 ts_tablet_manager.cc:1367] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.009s
I20240502 21:29:02.317349 11797 raft_consensus.cc:348] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.317660 11797 raft_consensus.cc:374] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.317804 11797 raft_consensus.cc:729] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.318132 11797 consensus_queue.cc:260] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.318369 11799 raft_consensus.cc:348] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.318605 11799 raft_consensus.cc:374] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.318681 11799 raft_consensus.cc:729] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.318996 11799 consensus_queue.cc:260] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.320011 11803 raft_consensus.cc:504] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.321170 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.321432 11605 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.321782 11797 ts_tablet_manager.cc:1398] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240502 21:29:02.322252 11799 ts_tablet_manager.cc:1398] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240502 21:29:02.324031 11797 tablet_bootstrap.cc:492] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.324764 11605 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.324913 11799 tablet_bootstrap.cc:492] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.325214 11718 leader_election.cc:302] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.325951 11797 tablet_bootstrap.cc:654] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.326226 11829 raft_consensus.cc:2781] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.326743 11803 leader_election.cc:288] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.326968 11799 tablet_bootstrap.cc:654] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.331329 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.331833 11681 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.332448 11680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.332782 11680 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.333309 11798 tablet_bootstrap.cc:492] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.333535 11798 ts_tablet_manager.cc:1367] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.006s
I20240502 21:29:02.334726 11798 raft_consensus.cc:348] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.335043 11798 raft_consensus.cc:374] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.335230 11798 raft_consensus.cc:729] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.335588 11798 consensus_queue.cc:260] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.336062 11805 raft_consensus.cc:482] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.336288 11805 raft_consensus.cc:504] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.336611 11798 ts_tablet_manager.cc:1398] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 21:29:02.337096 11798 tablet_bootstrap.cc:492] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a: Bootstrap starting.
I20240502 21:29:02.337797 11805 leader_election.cc:288] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.341099 11798 tablet_bootstrap.cc:654] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.341394 11680 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.343691 11797 tablet_bootstrap.cc:492] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.343900 11797 ts_tablet_manager.cc:1367] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.004s
I20240502 21:29:02.345451 11797 raft_consensus.cc:348] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.345815 11797 raft_consensus.cc:374] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.345937 11797 raft_consensus.cc:729] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.346282 11797 consensus_queue.cc:260] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.348431 11804 raft_consensus.cc:482] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.348668 11804 raft_consensus.cc:504] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.349725 11804 leader_election.cc:288] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.350133 11797 ts_tablet_manager.cc:1398] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 21:29:02.350813 11756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
I20240502 21:29:02.351181 11756 raft_consensus.cc:2454] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 0.
I20240502 21:29:02.351774 11647 leader_election.cc:302] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.352630 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.352941 11605 raft_consensus.cc:2454] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 0.
I20240502 21:29:02.353782 11804 raft_consensus.cc:2781] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.353965 11804 raft_consensus.cc:482] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.354108 11804 raft_consensus.cc:3037] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.355708 11797 tablet_bootstrap.cc:492] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.357988 11756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
I20240502 21:29:02.358276 11756 raft_consensus.cc:2454] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 0.
I20240502 21:29:02.358987 11570 leader_election.cc:302] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.359454 11805 raft_consensus.cc:2781] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.359599 11805 raft_consensus.cc:482] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.359725 11805 raft_consensus.cc:3037] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.359934 11803 raft_consensus.cc:482] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.360222 11803 raft_consensus.cc:504] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.361058 11803 leader_election.cc:288] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.359736 11797 tablet_bootstrap.cc:654] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.364182 11680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.366469 11829 raft_consensus.cc:686] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.367262 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.368130 11799 tablet_bootstrap.cc:492] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.368662 11605 raft_consensus.cc:2454] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
W20240502 21:29:02.369026 11719 leader_election.cc:341] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:02.370069 11799 ts_tablet_manager.cc:1367] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.044s user 0.020s sys 0.016s
I20240502 21:29:02.370406 11805 raft_consensus.cc:504] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.372174 11804 raft_consensus.cc:504] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.373198 11718 leader_election.cc:302] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters: 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:02.373729 11803 raft_consensus.cc:2781] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.373766 11804 leader_election.cc:288] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.374496 11803 raft_consensus.cc:482] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.376055 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.375825 11805 leader_election.cc:288] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.376230 11803 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.376688 11756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.377045 11804 raft_consensus.cc:482] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.377346 11756 raft_consensus.cc:3037] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.376737 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.372148 11829 consensus_queue.cc:237] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.378221 11605 raft_consensus.cc:2379] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 47a8645e1fc349299fd909d30cf01dd1 in current term 1: Already voted for candidate fa0b4ce7618d42ff8a424b77f59a89cd in this term.
I20240502 21:29:02.379570 11799 raft_consensus.cc:348] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.379076 11804 raft_consensus.cc:504] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.381829 11799 raft_consensus.cc:374] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.382607 11644 leader_election.cc:302] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1; no voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.383368 11799 raft_consensus.cc:729] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.383715 11799 consensus_queue.cc:260] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.384955 11804 leader_election.cc:288] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.385358 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.385699 11798 tablet_bootstrap.cc:492] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a: No bootstrap required, opened a new log
I20240502 21:29:02.385797 11830 raft_consensus.cc:482] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.386554 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
I20240502 21:29:02.386816 11804 raft_consensus.cc:2726] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 21:29:02.387598 11605 raft_consensus.cc:2454] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 0.
I20240502 21:29:02.385973 11830 raft_consensus.cc:504] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.388067 11644 leader_election.cc:302] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.391628 11798 ts_tablet_manager.cc:1367] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent bootstrapping tablet: real 0.049s user 0.015s sys 0.001s
I20240502 21:29:02.393029 11755 raft_consensus.cc:2454] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 0.
I20240502 21:29:02.393357 11799 ts_tablet_manager.cc:1398] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20240502 21:29:02.395109 11799 tablet_bootstrap.cc:492] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: Bootstrap starting.
I20240502 21:29:02.395109 11804 raft_consensus.cc:2781] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.396308 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.396981 11804 raft_consensus.cc:482] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.397330 11680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.397775 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.398660 11804 raft_consensus.cc:3037] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.398762 11681 raft_consensus.cc:2379] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa0b4ce7618d42ff8a424b77f59a89cd in current term 1: Already voted for candidate 47a8645e1fc349299fd909d30cf01dd1 in this term.
I20240502 21:29:02.398833 11680 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.397744 11803 raft_consensus.cc:504] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.398492 11605 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.399729 11719 leader_election.cc:302] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.396983 11830 leader_election.cc:288] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.400719 11756 raft_consensus.cc:2454] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 1.
I20240502 21:29:02.403034 11829 raft_consensus.cc:2781] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.399755 11679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.403687 11570 leader_election.cc:302] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.403916 11803 leader_election.cc:288] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.406025 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.406220 11805 raft_consensus.cc:2781] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.406221 11679 raft_consensus.cc:2379] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa0b4ce7618d42ff8a424b77f59a89cd in current term 1: Already voted for candidate 47a8645e1fc349299fd909d30cf01dd1 in this term.
I20240502 21:29:02.406869 11798 raft_consensus.cc:348] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.407223 11605 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.407269 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.409070 11798 raft_consensus.cc:374] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.409160 11829 raft_consensus.cc:482] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20240502 21:29:02.409541 11719 leader_election.cc:341] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:02.410809 11798 raft_consensus.cc:729] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.411278 11797 tablet_bootstrap.cc:492] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.411514 11797 ts_tablet_manager.cc:1367] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.058s user 0.006s sys 0.001s
I20240502 21:29:02.411815 11829 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.412129 11799 tablet_bootstrap.cc:654] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.412778 11804 raft_consensus.cc:504] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.412757 11499 catalog_manager.cc:5484] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 1 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:02.414945 11798 consensus_queue.cc:260] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.416781 11804 leader_election.cc:288] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.416747 11805 raft_consensus.cc:686] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:02.416741 11797 raft_consensus.cc:348] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.417343 11797 raft_consensus.cc:374] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.420219 11797 raft_consensus.cc:729] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.420063 11805 consensus_queue.cc:237] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.420127 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.420485 11605 raft_consensus.cc:2454] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.420562 11604 raft_consensus.cc:3037] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.420897 11797 consensus_queue.cc:260] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.421741 11797 ts_tablet_manager.cc:1398] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.010s user 0.001s sys 0.002s
I20240502 21:29:02.422112 11797 tablet_bootstrap.cc:492] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1: Bootstrap starting.
I20240502 21:29:02.423862 11839 raft_consensus.cc:482] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.424193 11797 tablet_bootstrap.cc:654] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:02.424041 11839 raft_consensus.cc:504] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.425419 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0defef714aa44459a0a1a1a6468dfe0f" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
I20240502 21:29:02.425477 11829 raft_consensus.cc:504] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.425673 11755 raft_consensus.cc:2454] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 0.
I20240502 21:29:02.424168 11718 leader_election.cc:302] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters: 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:02.426535 11829 leader_election.cc:288] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.426615 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.426769 11829 raft_consensus.cc:2781] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.426864 11681 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.426955 11829 raft_consensus.cc:686] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.420161 11756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.427561 11756 raft_consensus.cc:3037] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.427572 11570 leader_election.cc:302] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.425546 11604 raft_consensus.cc:2454] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 1.
I20240502 21:29:02.428682 11829 consensus_queue.cc:237] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.416769 11798 ts_tablet_manager.cc:1398] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20240502 21:29:02.433357 11644 leader_election.cc:302] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.433816 11804 raft_consensus.cc:2781] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.434026 11804 raft_consensus.cc:686] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:02.434448 11804 consensus_queue.cc:237] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.433029 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.446729 11839 leader_election.cc:288] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.449801 11836 raft_consensus.cc:482] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.450069 11836 raft_consensus.cc:504] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.451490 11836 leader_election.cc:288] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.452204 11839 raft_consensus.cc:2781] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.453058 11605 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.453851 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.454367 11681 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.458418 11719 leader_election.cc:302] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.458874 11604 raft_consensus.cc:2454] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 0.
I20240502 21:29:02.459640 11839 raft_consensus.cc:482] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.459688 11644 leader_election.cc:302] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.459648 11803 raft_consensus.cc:2781] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.460456 11803 raft_consensus.cc:686] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.461005 11803 consensus_queue.cc:237] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.462797 11756 raft_consensus.cc:2454] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 1.
I20240502 21:29:02.459803 11839 raft_consensus.cc:3037] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.463351 11605 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.464254 11836 raft_consensus.cc:2781] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.464268 11797 tablet_bootstrap.cc:492] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1: No bootstrap required, opened a new log
I20240502 21:29:02.465688 11797 ts_tablet_manager.cc:1367] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1: Time spent bootstrapping tablet: real 0.043s user 0.017s sys 0.008s
I20240502 21:29:02.467015 11797 raft_consensus.cc:348] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.467379 11797 raft_consensus.cc:374] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.467509 11797 raft_consensus.cc:729] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.467918 11797 consensus_queue.cc:260] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.468773 11797 ts_tablet_manager.cc:1398] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:02.470024 11836 raft_consensus.cc:482] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.474171 11836 raft_consensus.cc:3037] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.474331 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0defef714aa44459a0a1a1a6468dfe0f" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.474644 11681 raft_consensus.cc:2454] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 0.
I20240502 21:29:02.475903 11499 catalog_manager.cc:5484] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 0 to 1, leader changed from <none> to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 1 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.475912 11501 catalog_manager.cc:5484] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 1 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.477383 11839 raft_consensus.cc:504] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.478193 11756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
I20240502 21:29:02.478298 11836 raft_consensus.cc:504] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.479573 11839 leader_election.cc:288] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.481406 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0defef714aa44459a0a1a1a6468dfe0f" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.481738 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0defef714aa44459a0a1a1a6468dfe0f" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.481743 11681 raft_consensus.cc:3037] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.482909 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.483045 11756 raft_consensus.cc:2454] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 0.
I20240502 21:29:02.483126 11605 raft_consensus.cc:3037] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.483493 11836 leader_election.cc:288] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.484246 11755 raft_consensus.cc:3037] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.483918 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.485301 11754 raft_consensus.cc:3037] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.487584 11799 tablet_bootstrap.cc:492] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: No bootstrap required, opened a new log
I20240502 21:29:02.488474 11799 ts_tablet_manager.cc:1367] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent bootstrapping tablet: real 0.094s user 0.017s sys 0.017s
I20240502 21:29:02.489377 11501 catalog_manager.cc:5484] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 1 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:02.489854 11799 raft_consensus.cc:348] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.490132 11799 raft_consensus.cc:374] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:02.490238 11799 raft_consensus.cc:729] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Initialized, Role: FOLLOWER
I20240502 21:29:02.490554 11799 consensus_queue.cc:260] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.491144 11799 ts_tablet_manager.cc:1398] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:02.492038 11605 raft_consensus.cc:2454] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 1.
I20240502 21:29:02.492627 11644 leader_election.cc:302] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.492962 11836 raft_consensus.cc:2781] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.493124 11836 raft_consensus.cc:686] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:02.493438 11836 consensus_queue.cc:237] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.494307 11755 raft_consensus.cc:2454] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 1.
I20240502 21:29:02.494764 11570 leader_election.cc:302] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.494969 11754 raft_consensus.cc:2454] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 1.
I20240502 21:29:02.495502 11839 raft_consensus.cc:2781] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.495775 11839 raft_consensus.cc:686] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:02.496426 11839 consensus_queue.cc:237] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.496984 11681 raft_consensus.cc:2454] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 1.
I20240502 21:29:02.499505 11499 catalog_manager.cc:5484] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 1 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:02.502912 11501 catalog_manager.cc:5484] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 1 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.504339 11500 catalog_manager.cc:5484] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 0 to 1, leader changed from <none> to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 1 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.534873 11829 raft_consensus.cc:482] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.535163 11829 raft_consensus.cc:504] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.535873 11829 leader_election.cc:288] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.536478 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.536475 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.536783 11605 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.536844 11681 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.537274 11718 leader_election.cc:302] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.537598 11829 raft_consensus.cc:2781] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.537722 11829 raft_consensus.cc:482] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.537842 11829 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.539845 11829 raft_consensus.cc:504] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.540648 11829 leader_election.cc:288] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.541148 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.541153 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.541389 11605 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.541443 11681 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.543699 11605 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.543704 11681 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.544281 11718 leader_election.cc:302] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.544665 11829 raft_consensus.cc:2781] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.544862 11829 raft_consensus.cc:686] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.545176 11829 consensus_queue.cc:237] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.548009 11500 catalog_manager.cc:5484] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 1 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:02.640517 11829 raft_consensus.cc:482] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.640821 11829 raft_consensus.cc:504] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.641590 11829 leader_election.cc:288] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.642244 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.642293 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.642514 11681 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.642578 11605 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.643057 11719 leader_election.cc:302] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.643504 11829 raft_consensus.cc:2781] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.643648 11829 raft_consensus.cc:482] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.643749 11829 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.645960 11829 raft_consensus.cc:504] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.646888 11829 leader_election.cc:288] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.647482 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.647573 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.647761 11681 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.647823 11605 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.650130 11681 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.650213 11605 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.650756 11719 leader_election.cc:302] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.651118 11829 raft_consensus.cc:2781] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.651254 11829 raft_consensus.cc:686] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.651567 11829 consensus_queue.cc:237] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:02.654956 11500 catalog_manager.cc:5484] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 1 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.673779 11829 raft_consensus.cc:482] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.674072 11829 raft_consensus.cc:504] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.674849 11829 leader_election.cc:288] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.675475 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.675477 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1" is_pre_election: true
I20240502 21:29:02.675817 11605 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.675880 11681 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 0.
I20240502 21:29:02.676424 11718 leader_election.cc:302] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.677196 11829 raft_consensus.cc:2781] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:02.677403 11829 raft_consensus.cc:482] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:02.677597 11829 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.681677 11829 raft_consensus.cc:504] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.682596 11829 leader_election.cc:288] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.684679 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.684679 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.685029 11605 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.685087 11681 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.688480 11605 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.688480 11681 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 1.
I20240502 21:29:02.689093 11719 leader_election.cc:302] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.689461 11829 raft_consensus.cc:2781] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:02.689653 11829 raft_consensus.cc:686] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.690011 11829 consensus_queue.cc:237] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.693194 11500 catalog_manager.cc:5484] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 1 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:02.748726 11836 raft_consensus.cc:482] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:02.749039 11836 raft_consensus.cc:504] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.749804 11836 leader_election.cc:288] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.750344 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" is_pre_election: true
I20240502 21:29:02.750514 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" is_pre_election: true
I20240502 21:29:02.750655 11605 raft_consensus.cc:2379] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 47a8645e1fc349299fd909d30cf01dd1 in current term 1: Already voted for candidate 152ea2e21dd1407586e9110cfc5e7d5a in this term.
I20240502 21:29:02.751174 11647 leader_election.cc:302] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1; no voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.751588 11836 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:02.754328 11836 raft_consensus.cc:2726] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:02.759737 11755 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:02.760064 11755 raft_consensus.cc:595] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.763010 11755 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:02.763341 11755 raft_consensus.cc:595] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.766064 11755 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "272a4fe221da43439bc128eb339ef434"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:02.766315 11755 raft_consensus.cc:595] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.772657 11755 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9a990045515e4479b625327a285c6af3"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:02.772920 11755 raft_consensus.cc:595] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Received request to transfer leadership to TS 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:02.775384 11755 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:02.775619 11755 raft_consensus.cc:595] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Received request to transfer leadership to TS 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:02.777910 11755 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:02.778136 11755 raft_consensus.cc:595] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Received request to transfer leadership to TS 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:02.785828 11681 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
mode: GRACEFUL
new_leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:36648
I20240502 21:29:02.786069 11681 raft_consensus.cc:595] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Received request to transfer leadership to TS 152ea2e21dd1407586e9110cfc5e7d5a
I20240502 21:29:02.816779 11829 consensus_queue.cc:1035] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.819911 11803 raft_consensus.cc:979] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:02.820168 11803 raft_consensus.cc:1067] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Signalling peer 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:02.820955 11679 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:36628
I20240502 21:29:02.821841 11679 raft_consensus.cc:482] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:02.822014 11679 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.823604 11803 consensus_queue.cc:1035] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.825035 11679 raft_consensus.cc:504] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.826486 11679 leader_election.cc:288] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.827041 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.827780 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.828202 11755 raft_consensus.cc:3032] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:02.828321 11755 raft_consensus.cc:729] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.828632 11755 consensus_queue.cc:260] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.828922 11755 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.831053 11755 raft_consensus.cc:2454] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:02.831676 11647 leader_election.cc:302] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters: fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:02.832038 11836 raft_consensus.cc:2781] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:02.832264 11836 raft_consensus.cc:686] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:02.832581 11836 consensus_queue.cc:237] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.835193 11500 catalog_manager.cc:5484] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 1 to 2, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 2 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.851239 11803 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.854485 11829 raft_consensus.cc:979] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:02.854694 11830 raft_consensus.cc:1067] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:02.855511 11604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "272a4fe221da43439bc128eb339ef434"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44092
I20240502 21:29:02.856007 11604 raft_consensus.cc:482] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:02.856199 11604 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.856521 11803 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.859495 11604 raft_consensus.cc:504] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.860918 11604 leader_election.cc:288] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.862418 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.862699 11755 raft_consensus.cc:3032] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:02.862810 11755 raft_consensus.cc:729] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.863160 11755 consensus_queue.cc:260] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.863603 11755 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.864208 11679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.864430 11679 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.865885 11755 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:02.866353 11570 leader_election.cc:302] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.866631 11679 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:02.866729 11839 raft_consensus.cc:2781] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:02.866937 11839 raft_consensus.cc:686] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:02.867276 11839 consensus_queue.cc:237] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.869845 11501 catalog_manager.cc:5484] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 1 to 2, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 2 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.885288 11836 consensus_queue.cc:1035] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.890062 11836 consensus_queue.cc:1035] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.890187 11804 consensus_queue.cc:1035] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.893798 11839 consensus_queue.cc:1035] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.896359 11804 consensus_queue.cc:1035] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.901525 11804 raft_consensus.cc:979] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: : Instructing follower 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:02.901716 11804 raft_consensus.cc:1067] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Signalling peer 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:02.902370 11755 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:54180
I20240502 21:29:02.902287 11840 consensus_queue.cc:1035] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.902594 11755 raft_consensus.cc:482] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:02.902714 11755 raft_consensus.cc:3037] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.906695 11755 raft_consensus.cc:504] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.907807 11755 leader_election.cc:288] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 2 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:02.908308 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:02.908532 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da63340e627438f895d2a7edd31e5c7" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.908622 11605 raft_consensus.cc:3037] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.908789 11681 raft_consensus.cc:3032] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:02.908921 11681 raft_consensus.cc:729] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:02.909260 11681 consensus_queue.cc:260] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.909674 11681 raft_consensus.cc:3037] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.912221 11681 raft_consensus.cc:2454] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 2.
I20240502 21:29:02.912767 11719 leader_election.cc:302] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:02.913100 11605 raft_consensus.cc:2454] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 2.
I20240502 21:29:02.913179 11803 raft_consensus.cc:2781] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:02.913424 11803 raft_consensus.cc:686] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.913786 11803 consensus_queue.cc:237] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.916811 11501 catalog_manager.cc:5484] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 1 to 2, leader changed from 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66) to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 2 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:02.937479 11803 consensus_queue.cc:1035] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.940559 11830 raft_consensus.cc:979] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:02.940797 11830 raft_consensus.cc:1067] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:02.941514 11604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44092
I20240502 21:29:02.941787 11604 raft_consensus.cc:482] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:02.941926 11604 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.943403 11803 consensus_queue.cc:1035] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.945147 11604 raft_consensus.cc:504] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.946466 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:02.946727 11681 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.947085 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:02.947290 11755 raft_consensus.cc:3032] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:02.947451 11604 leader_election.cc:288] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:02.948279 11755 raft_consensus.cc:729] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:02.948663 11755 consensus_queue.cc:260] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.949132 11755 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:02.949692 11681 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:02.950210 11568 leader_election.cc:302] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:02.950546 11840 raft_consensus.cc:2781] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:02.950748 11840 raft_consensus.cc:686] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:02.951095 11840 consensus_queue.cc:237] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:02.953861 11500 catalog_manager.cc:5484] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 1 to 2, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 2 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:02.954617 11755 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:02.995213 11840 consensus_queue.cc:1035] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:02.999469 11839 consensus_queue.cc:1035] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.059708 11803 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.065169 11803 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.068318 11830 raft_consensus.cc:979] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:03.068528 11830 raft_consensus.cc:1067] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Signalling peer 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:03.069265 11681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9a990045515e4479b625327a285c6af3"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:36628
I20240502 21:29:03.069497 11681 raft_consensus.cc:482] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:03.069638 11681 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.071802 11681 raft_consensus.cc:504] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:03.072504 11681 leader_election.cc:288] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:03.072966 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:03.073024 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:03.073177 11604 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.073205 11755 raft_consensus.cc:3032] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:03.073346 11755 raft_consensus.cc:729] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:03.073618 11755 consensus_queue.cc:260] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:03.073972 11755 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.075562 11604 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:03.076123 11644 leader_election.cc:302] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:03.076306 11755 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:03.076469 11804 raft_consensus.cc:2781] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:03.076634 11804 raft_consensus.cc:686] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:03.076951 11804 consensus_queue.cc:237] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:03.079582 11501 catalog_manager.cc:5484] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 1 to 2, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 2 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:03.134383 11830 consensus_queue.cc:1035] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.137897 11803 raft_consensus.cc:979] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:03.138147 11803 raft_consensus.cc:1067] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:03.139565 11604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44092
I20240502 21:29:03.140077 11604 raft_consensus.cc:482] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:03.140285 11604 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.140530 11803 consensus_queue.cc:1035] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.144145 11604 raft_consensus.cc:504] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:03.145463 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:03.145604 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:03.145728 11755 raft_consensus.cc:3032] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:03.145841 11681 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.145898 11604 leader_election.cc:288] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Requested vote from peers 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:03.145864 11755 raft_consensus.cc:729] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:03.146447 11755 consensus_queue.cc:260] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:03.146878 11755 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.148782 11681 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:03.149237 11568 leader_election.cc:302] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:03.149358 11755 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:03.149585 11839 raft_consensus.cc:2781] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:03.149742 11839 raft_consensus.cc:686] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:03.150043 11839 consensus_queue.cc:237] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:03.152629 11500 catalog_manager.cc:5484] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 1 to 2, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 2 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:03.240196 11803 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.245362 11829 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:03.249022 11830 raft_consensus.cc:979] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:03.249215 11830 raft_consensus.cc:1067] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Signalling peer 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:03.249855 11681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:36628
I20240502 21:29:03.250087 11681 raft_consensus.cc:482] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:03.250221 11681 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.253129 11681 raft_consensus.cc:504] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:03.253595 11604 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.253882 11681 leader_election.cc:288] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:03.254371 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:03.254628 11605 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.256260 11755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:03.256489 11755 raft_consensus.cc:3032] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:03.256618 11755 raft_consensus.cc:729] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:03.256908 11755 consensus_queue.cc:260] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:03.257265 11755 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:03.257238 11604 raft_consensus.cc:1261] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.257905 11605 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:03.258112 11804 consensus_queue.cc:1035] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.258375 11644 leader_election.cc:302] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:03.258702 11804 raft_consensus.cc:2781] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:03.258886 11804 raft_consensus.cc:686] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:03.259322 11804 consensus_queue.cc:237] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:03.262090 11754 raft_consensus.cc:1261] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.262166 11500 catalog_manager.cc:5484] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 1 to 2, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 2 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:03.262691 11804 consensus_queue.cc:1035] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.265262 11755 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:03.296038 11605 raft_consensus.cc:1261] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.296741 11830 consensus_queue.cc:1035] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.300299 11681 raft_consensus.cc:1261] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.300940 11829 consensus_queue.cc:1035] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.303627 11681 raft_consensus.cc:1261] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.304440 11839 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.308488 11755 raft_consensus.cc:1261] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.309027 11878 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.331707 11681 raft_consensus.cc:1261] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.332477 11878 consensus_queue.cc:1035] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.335855 11755 raft_consensus.cc:1261] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.336982 11839 consensus_queue.cc:1035] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.664801 11755 raft_consensus.cc:1261] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.665560 11804 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.667388 11605 raft_consensus.cc:1261] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.667908 11804 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.706247 11605 raft_consensus.cc:1261] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.706890 11845 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.711112 11755 raft_consensus.cc:1261] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.711761 11836 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.723611 11755 raft_consensus.cc:1261] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.724325 11878 consensus_queue.cc:1035] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:03.729084 11681 raft_consensus.cc:1261] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:03.729820 11878 consensus_queue.cc:1035] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:05.809553 10981 auto_leader_rebalancer-test.cc:331] The leader distribution is
I20240502 21:29:05.809800 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:05.817934 11681 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
mode: GRACEFUL
new_leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:59834
I20240502 21:29:05.818171 11681 raft_consensus.cc:595] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Received request to transfer leadership to TS 152ea2e21dd1407586e9110cfc5e7d5a
I20240502 21:29:05.824467 11605 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "272a4fe221da43439bc128eb339ef434"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
mode: GRACEFUL
new_leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:39888
I20240502 21:29:05.824707 11605 raft_consensus.cc:595] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Received request to transfer leadership to TS 152ea2e21dd1407586e9110cfc5e7d5a
I20240502 21:29:05.825241 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240502 21:29:05.825460 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:06.104928 11878 raft_consensus.cc:979] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: : Instructing follower 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:06.105142 11878 raft_consensus.cc:1067] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Signalling peer 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:06.106014 11755 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "272a4fe221da43439bc128eb339ef434"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:54192
I20240502 21:29:06.106321 11755 raft_consensus.cc:482] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:06.106500 11755 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:06.108675 11755 raft_consensus.cc:504] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:06.109447 11755 leader_election.cc:288] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:06.110030 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:06.110174 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:06.110360 11681 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:06.110369 11605 raft_consensus.cc:3032] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:29:06.110502 11605 raft_consensus.cc:729] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:06.110827 11605 consensus_queue.cc:260] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:06.111295 11605 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:06.113664 11681 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:06.114223 11605 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:06.114308 11719 leader_election.cc:302] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:06.114675 11882 raft_consensus.cc:2781] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:29:06.115132 11882 raft_consensus.cc:686] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:06.115445 11882 consensus_queue.cc:237] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:06.118546 11500 catalog_manager.cc:5484] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 2 to 3, leader changed from fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65) to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 3 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:06.273499 11885 raft_consensus.cc:979] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: : Instructing follower 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:06.273751 11885 raft_consensus.cc:1067] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Signalling peer 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:06.274500 11755 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:54180
I20240502 21:29:06.274691 11755 raft_consensus.cc:482] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:06.274799 11755 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:06.276769 11755 raft_consensus.cc:504] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:06.277369 11755 leader_election.cc:288] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:06.277957 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:06.278081 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:06.278167 11605 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:06.278331 11681 raft_consensus.cc:3032] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:29:06.278419 11681 raft_consensus.cc:729] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:06.278725 11681 consensus_queue.cc:260] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:06.279075 11681 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:06.280761 11605 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:06.281212 11718 leader_election.cc:302] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:06.281543 11882 raft_consensus.cc:2781] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:29:06.281749 11681 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:06.281783 11882 raft_consensus.cc:686] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:06.282092 11882 consensus_queue.cc:237] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:06.284799 11500 catalog_manager.cc:5484] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 2 to 3, leader changed from 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66) to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 3 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:06.675285 11605 raft_consensus.cc:1261] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:06.676120 11882 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:06.681262 11681 raft_consensus.cc:1261] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:06.682018 11890 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:06.825910 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:06.827252 11605 raft_consensus.cc:1261] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:06.827951 11890 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:06.828614 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:06.828878 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:06.832082 11681 raft_consensus.cc:1261] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:06.832780 11890 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:07.829401 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:07.831416 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:07.831665 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:08.832170 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:08.833945 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:08.834172 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:09.834627 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:09.836714 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:09.836930 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:10.837365 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:10.839447 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:10.839699 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:11.840246 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:11.842078 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:11.842296 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:12.842829 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:12.844799 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:12.845033 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:13.845508 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:13.847690 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:13.847890 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:14.848299 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:14.850085 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:14.850281 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:15.863487 10981 auto_leader_rebalancer-test.cc:350] The leader distribution is
I20240502 21:29:15.867748 11754 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "272a4fe221da43439bc128eb339ef434"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:15.868036 11754 raft_consensus.cc:595] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:15.872359 11681 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:36648
I20240502 21:29:15.872637 11681 raft_consensus.cc:595] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:15.874720 11681 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9a990045515e4479b625327a285c6af3"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:36648
I20240502 21:29:15.874963 11681 raft_consensus.cc:595] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:15.876976 11681 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
mode: GRACEFUL
new_leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:36648
I20240502 21:29:15.877207 11681 raft_consensus.cc:595] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Received request to transfer leadership to TS fa0b4ce7618d42ff8a424b77f59a89cd
I20240502 21:29:15.879474 11754 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
mode: GRACEFUL
new_leader_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:54206
I20240502 21:29:15.879668 11754 raft_consensus.cc:595] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Received request to transfer leadership to TS 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:15.962812 11910 raft_consensus.cc:979] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:15.963094 11885 raft_consensus.cc:1067] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:15.963862 11605 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44090
I20240502 21:29:15.964107 11605 raft_consensus.cc:482] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:15.964241 11605 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:15.966647 11605 raft_consensus.cc:504] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:15.967281 11605 leader_election.cc:288] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 3 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:15.967797 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:15.967803 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8839111f2ddd4ae28be77b16966e659f" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:15.968063 11681 raft_consensus.cc:3032] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:29:15.968075 11754 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:15.968159 11681 raft_consensus.cc:729] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:15.968454 11681 consensus_queue.cc:260] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:15.968792 11681 raft_consensus.cc:3037] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:15.970376 11754 raft_consensus.cc:2454] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 3.
I20240502 21:29:15.970840 11570 leader_election.cc:302] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:15.970947 11681 raft_consensus.cc:2454] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 3.
I20240502 21:29:15.971150 11906 raft_consensus.cc:2781] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:29:15.971580 11906 raft_consensus.cc:686] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:15.971940 11906 consensus_queue.cc:237] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:15.974609 11499 catalog_manager.cc:5484] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 2 to 3, leader changed from 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 3 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:15.984210 11885 raft_consensus.cc:979] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:15.984395 11885 raft_consensus.cc:1067] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:15.985054 11605 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44090
I20240502 21:29:15.985252 11605 raft_consensus.cc:482] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:15.985343 11605 raft_consensus.cc:3037] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:15.987321 11605 raft_consensus.cc:504] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:15.987938 11605 leader_election.cc:288] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:15.988458 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:15.988497 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df366396c6044cf4b642bcc2da2046b3" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:15.988672 11754 raft_consensus.cc:3037] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:15.988754 11681 raft_consensus.cc:3032] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:15.988853 11681 raft_consensus.cc:729] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:15.989162 11681 consensus_queue.cc:260] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:15.989524 11681 raft_consensus.cc:3037] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:15.990844 11754 raft_consensus.cc:2454] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:15.991253 11570 leader_election.cc:302] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:15.991576 11906 raft_consensus.cc:2781] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:15.991755 11906 raft_consensus.cc:686] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:15.991784 11681 raft_consensus.cc:2454] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 2.
I20240502 21:29:15.992130 11906 consensus_queue.cc:237] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:15.994921 11499 catalog_manager.cc:5484] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 1 to 2, leader changed from 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 2 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:16.066668 11903 raft_consensus.cc:979] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:16.066862 11903 raft_consensus.cc:1067] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Signalling peer 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:16.067610 11681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:36628
I20240502 21:29:16.067859 11681 raft_consensus.cc:482] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:16.068003 11681 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:16.070065 11681 raft_consensus.cc:504] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.070730 11681 leader_election.cc:288] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 4 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:16.071322 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:16.071373 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "085e2f4b603744e1bf1c1394a54dc9bc" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:16.071525 11754 raft_consensus.cc:3032] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Stepping down as leader of term 3
I20240502 21:29:16.071544 11605 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:16.071678 11754 raft_consensus.cc:729] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:16.072054 11754 consensus_queue.cc:260] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.072605 11754 raft_consensus.cc:3037] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:16.074213 11605 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 4.
I20240502 21:29:16.074657 11644 leader_election.cc:302] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:16.074838 11754 raft_consensus.cc:2454] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 4.
I20240502 21:29:16.074982 11885 raft_consensus.cc:2781] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 21:29:16.075174 11885 raft_consensus.cc:686] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 4 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:16.075562 11885 consensus_queue.cc:237] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.079423 11499 catalog_manager.cc:5484] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 3 to 4, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 4 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:16.163813 11918 raft_consensus.cc:979] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:16.164139 11903 raft_consensus.cc:1067] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:16.164810 11605 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "272a4fe221da43439bc128eb339ef434"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44092
I20240502 21:29:16.165042 11605 raft_consensus.cc:482] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:16.165161 11605 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:16.167165 11605 raft_consensus.cc:504] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.167829 11605 leader_election.cc:288] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 4 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:16.168311 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:16.168342 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272a4fe221da43439bc128eb339ef434" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:16.168603 11754 raft_consensus.cc:3032] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Stepping down as leader of term 3
I20240502 21:29:16.168601 11681 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:16.168717 11754 raft_consensus.cc:729] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:16.168994 11754 consensus_queue.cc:260] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.169337 11754 raft_consensus.cc:3037] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:16.170884 11681 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 4.
I20240502 21:29:16.171319 11568 leader_election.cc:302] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a8645e1fc349299fd909d30cf01dd1, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:16.171414 11754 raft_consensus.cc:2454] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 4.
I20240502 21:29:16.171641 11906 raft_consensus.cc:2781] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Leader election won for term 4
I20240502 21:29:16.171811 11906 raft_consensus.cc:686] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:16.172118 11906 consensus_queue.cc:237] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.174803 11499 catalog_manager.cc:5484] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 3 to 4, leader changed from 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 4 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:16.369607 11681 raft_consensus.cc:1261] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:16.370263 11906 consensus_queue.cc:1035] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:16.374837 11754 raft_consensus.cc:1261] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:16.375623 11911 consensus_queue.cc:1035] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:16.397383 11910 raft_consensus.cc:979] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: : Instructing follower fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:16.397610 11910 raft_consensus.cc:1067] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Signalling peer fa0b4ce7618d42ff8a424b77f59a89cd to start an election
I20240502 21:29:16.398543 11605 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9a990045515e4479b625327a285c6af3"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
from {username='slave'} at 127.0.0.1:44090
I20240502 21:29:16.398787 11605 raft_consensus.cc:482] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:16.398931 11605 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:16.401162 11605 raft_consensus.cc:504] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.401852 11605 leader_election.cc:288] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 3 election: Requested vote from peers 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:16.402518 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:16.402518 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:16.402827 11681 raft_consensus.cc:3032] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:29:16.402828 11754 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:16.402992 11681 raft_consensus.cc:729] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:16.403388 11681 consensus_queue.cc:260] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.403877 11681 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:16.406355 11754 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 3.
I20240502 21:29:16.406848 11681 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa0b4ce7618d42ff8a424b77f59a89cd in term 3.
I20240502 21:29:16.406888 11570 leader_election.cc:302] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:16.407246 11911 raft_consensus.cc:2781] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:29:16.407509 11911 raft_consensus.cc:686] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Becoming Leader. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:16.407843 11911 consensus_queue.cc:237] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:16.410866 11500 catalog_manager.cc:5484] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd reported cstate change: term changed from 2 to 3, leader changed from 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66) to fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65). New cstate: current_term: 3 leader_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:16.584985 11681 raft_consensus.cc:1261] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:16.585691 11911 consensus_queue.cc:1035] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:16.588477 11754 raft_consensus.cc:1261] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:16.589224 11906 consensus_queue.cc:1035] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:16.650110 11754 raft_consensus.cc:1261] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:29:16.650848 11910 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:16.655618 11605 raft_consensus.cc:1261] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:29:16.656464 11885 consensus_queue.cc:1035] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:16.658891 11754 raft_consensus.cc:1261] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:29:16.659623 11906 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:16.665382 11681 raft_consensus.cc:1261] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:29:16.666225 11906 consensus_queue.cc:1035] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:16.815268 11681 raft_consensus.cc:1261] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:16.816004 11932 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:16.820619 11754 raft_consensus.cc:1261] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Refusing update from remote peer fa0b4ce7618d42ff8a424b77f59a89cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:16.821396 11906 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:18.892868 10981 auto_leader_rebalancer-test.cc:365] The leader distribution is
I20240502 21:29:18.893168 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:18.896240 11605 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
mode: GRACEFUL
new_leader_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:39888
I20240502 21:29:18.896454 11605 raft_consensus.cc:595] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Received request to transfer leadership to TS 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:18.897469 11605 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
mode: GRACEFUL
new_leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:39888
I20240502 21:29:18.897631 11605 raft_consensus.cc:595] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Received request to transfer leadership to TS 152ea2e21dd1407586e9110cfc5e7d5a
I20240502 21:29:18.898491 11605 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
mode: GRACEFUL
new_leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:39888
I20240502 21:29:18.898644 11605 raft_consensus.cc:595] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Received request to transfer leadership to TS 152ea2e21dd1407586e9110cfc5e7d5a
I20240502 21:29:18.899564 11605 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9a990045515e4479b625327a285c6af3"
dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
mode: GRACEFUL
new_leader_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:39888
I20240502 21:29:18.899722 11605 raft_consensus.cc:595] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Received request to transfer leadership to TS 47a8645e1fc349299fd909d30cf01dd1
I20240502 21:29:18.900125 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 4
I20240502 21:29:18.900353 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:19.083412 11937 raft_consensus.cc:979] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: : Instructing follower 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:19.083639 11911 raft_consensus.cc:1067] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Signalling peer 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:19.084501 11754 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:54192
I20240502 21:29:19.084699 11754 raft_consensus.cc:482] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:19.084808 11754 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:19.087918 11754 raft_consensus.cc:504] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:19.088965 11754 leader_election.cc:288] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:19.089491 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:19.089491 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d83bba66524f9288af49392ab4ed96" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:19.089766 11605 raft_consensus.cc:3032] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:29:19.089788 11681 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:19.089928 11605 raft_consensus.cc:729] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:19.090224 11605 consensus_queue.cc:260] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:19.090698 11605 raft_consensus.cc:3037] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:19.093070 11681 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:19.093583 11605 raft_consensus.cc:2454] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:19.093609 11719 leader_election.cc:302] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:19.093959 11903 raft_consensus.cc:2781] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:29:19.094375 11903 raft_consensus.cc:686] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:19.094719 11903 consensus_queue.cc:237] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } }
I20240502 21:29:19.097906 11499 catalog_manager.cc:5484] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 2 to 3, leader changed from fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65) to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 3 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:19.179636 11937 raft_consensus.cc:979] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: : Instructing follower 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:19.179881 11937 raft_consensus.cc:1067] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Signalling peer 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:19.180662 11681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9a990045515e4479b625327a285c6af3"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:36634
I20240502 21:29:19.180867 11681 raft_consensus.cc:482] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:19.181005 11681 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:19.183090 11681 raft_consensus.cc:504] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.184119 11681 leader_election.cc:288] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 4 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:19.184633 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:19.184736 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a990045515e4479b625327a285c6af3" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:19.184899 11754 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:19.184985 11605 raft_consensus.cc:3032] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Stepping down as leader of term 3
I20240502 21:29:19.185106 11605 raft_consensus.cc:729] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:19.185396 11605 consensus_queue.cc:260] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.185770 11605 raft_consensus.cc:3037] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Advancing to term 4
I20240502 21:29:19.187211 11754 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 4.
I20240502 21:29:19.187711 11647 leader_election.cc:302] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:19.187829 11605 raft_consensus.cc:2454] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 4.
I20240502 21:29:19.188055 11936 raft_consensus.cc:2781] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 21:29:19.188594 11936 raft_consensus.cc:686] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:19.188905 11936 consensus_queue.cc:237] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.191715 11499 catalog_manager.cc:5484] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 3 to 4, leader changed from fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65) to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 4 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:19.305655 11937 raft_consensus.cc:979] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: : Instructing follower 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:19.305862 11937 raft_consensus.cc:1067] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Signalling peer 152ea2e21dd1407586e9110cfc5e7d5a to start an election
I20240502 21:29:19.306547 11754 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d"
dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
from {username='slave'} at 127.0.0.1:54192
I20240502 21:29:19.306779 11754 raft_consensus.cc:482] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:19.306922 11754 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:19.309203 11754 raft_consensus.cc:504] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.309906 11754 leader_election.cc:288] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66:45427)
I20240502 21:29:19.310447 11681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
I20240502 21:29:19.310388 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b4317ce6a1b451a84ae2ced0fc8ae6d" candidate_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:19.310635 11605 raft_consensus.cc:3032] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Stepping down as leader of term 2
I20240502 21:29:19.310729 11605 raft_consensus.cc:729] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:19.310976 11605 consensus_queue.cc:260] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.311281 11605 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:19.313387 11911 raft_consensus.cc:979] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: : Instructing follower 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:19.313550 11911 raft_consensus.cc:1067] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Signalling peer 47a8645e1fc349299fd909d30cf01dd1 to start an election
I20240502 21:29:19.313730 11605 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:19.314039 11681 raft_consensus.cc:3037] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 21:29:19.314285 11679 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529"
dest_uuid: "47a8645e1fc349299fd909d30cf01dd1"
from {username='slave'} at 127.0.0.1:36634
I20240502 21:29:19.314405 11718 leader_election.cc:302] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, fa0b4ce7618d42ff8a424b77f59a89cd; no voters:
I20240502 21:29:19.314520 11679 raft_consensus.cc:482] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 21:29:19.314663 11679 raft_consensus.cc:3037] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:19.314823 11903 raft_consensus.cc:2781] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Leader election won for term 3
I20240502 21:29:19.314998 11903 raft_consensus.cc:686] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Becoming Leader. State: Replica: 152ea2e21dd1407586e9110cfc5e7d5a, State: Running, Role: LEADER
I20240502 21:29:19.315316 11903 consensus_queue.cc:237] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.317627 11681 raft_consensus.cc:2454] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152ea2e21dd1407586e9110cfc5e7d5a in term 3.
I20240502 21:29:19.317750 11679 raft_consensus.cc:504] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.318758 11679 leader_election.cc:288] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Requested vote from peers fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333), 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67:46519)
I20240502 21:29:19.319037 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd"
I20240502 21:29:19.319293 11605 raft_consensus.cc:3032] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 21:29:19.319420 11605 raft_consensus.cc:729] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa0b4ce7618d42ff8a424b77f59a89cd, State: Running, Role: LEADER
I20240502 21:29:19.319113 11499 catalog_manager.cc:5484] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a reported cstate change: term changed from 2 to 3, leader changed from fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65) to 152ea2e21dd1407586e9110cfc5e7d5a (127.10.185.67). New cstate: current_term: 3 leader_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:19.319677 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0059e9d031fa4b5c9ad9414f80a61529" candidate_uuid: "47a8645e1fc349299fd909d30cf01dd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "152ea2e21dd1407586e9110cfc5e7d5a"
I20240502 21:29:19.319710 11605 consensus_queue.cc:260] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.319907 11754 raft_consensus.cc:3037] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:19.320192 11605 raft_consensus.cc:3037] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Advancing to term 2
I20240502 21:29:19.323264 11754 raft_consensus.cc:2454] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:19.323264 11605 raft_consensus.cc:2454] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a8645e1fc349299fd909d30cf01dd1 in term 2.
I20240502 21:29:19.323725 11647 leader_election.cc:302] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152ea2e21dd1407586e9110cfc5e7d5a, 47a8645e1fc349299fd909d30cf01dd1; no voters:
I20240502 21:29:19.324010 11936 raft_consensus.cc:2781] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 21:29:19.324198 11936 raft_consensus.cc:686] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Becoming Leader. State: Replica: 47a8645e1fc349299fd909d30cf01dd1, State: Running, Role: LEADER
I20240502 21:29:19.324501 11936 consensus_queue.cc:237] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } }
I20240502 21:29:19.327044 11499 catalog_manager.cc:5484] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 reported cstate change: term changed from 1 to 2, leader changed from fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65) to 47a8645e1fc349299fd909d30cf01dd1 (127.10.185.66). New cstate: current_term: 2 leader_uuid: "47a8645e1fc349299fd909d30cf01dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:19.503654 11605 raft_consensus.cc:1261] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:19.504348 11903 consensus_queue.cc:1035] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:19.509418 11679 raft_consensus.cc:1261] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:19.510133 11903 consensus_queue.cc:1035] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:19.630746 11605 raft_consensus.cc:1261] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:29:19.631412 11936 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:19.635551 11754 raft_consensus.cc:1261] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240502 21:29:19.636166 11954 consensus_queue.cc:1035] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:19.709292 11605 raft_consensus.cc:1261] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:19.709957 11903 consensus_queue.cc:1035] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:19.714650 11679 raft_consensus.cc:1261] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Refusing update from remote peer 152ea2e21dd1407586e9110cfc5e7d5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240502 21:29:19.715698 11938 consensus_queue.cc:1035] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47a8645e1fc349299fd909d30cf01dd1" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:19.726714 11754 raft_consensus.cc:1261] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:19.727300 11954 consensus_queue.cc:1035] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152ea2e21dd1407586e9110cfc5e7d5a" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:19.731472 11605 raft_consensus.cc:1261] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Refusing update from remote peer 47a8645e1fc349299fd909d30cf01dd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 21:29:19.732085 11954 consensus_queue.cc:1035] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa0b4ce7618d42ff8a424b77f59a89cd" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:19.900846 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:19.902603 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:19.902822 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:20.903185 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:20.904964 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:20.905166 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:21.905673 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:21.907575 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:21.907779 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:22.908234 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:22.910286 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:22.910488 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:23.910889 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:23.912763 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:23.912946 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:24.913383 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:24.915207 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:24.915401 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:25.915838 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:25.917615 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:25.917804 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:26.918269 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:26.920104 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:26.920295 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:27.927162 10981 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240502 21:29:27.928997 10981 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240502 21:29:27.929235 10981 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240502 21:29:28.941414 10981 auto_leader_rebalancer-test.cc:376] The leader distribution is
I20240502 21:29:28.941689 10981 tablet_server.cc:178] TabletServer@127.10.185.65:0 shutting down...
W20240502 21:29:28.946027 11644 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20240502 21:29:28.948254 11718 consensus_peers.cc:485] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a -> Peer fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333): Couldn't send request to peer fa0b4ce7618d42ff8a424b77f59a89cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20240502 21:29:28.950459 11644 consensus_peers.cc:485] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 -> Peer fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333): Couldn't send request to peer fa0b4ce7618d42ff8a424b77f59a89cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:28.953611 10981 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240502 21:29:28.954002 10981 tablet_replica.cc:330] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.954221 10981 raft_consensus.cc:2229] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 LEADER]: Raft consensus shutting down.
I20240502 21:29:28.954597 10981 raft_consensus.cc:2258] T df366396c6044cf4b642bcc2da2046b3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.955206 10981 tablet_replica.cc:330] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.955384 10981 raft_consensus.cc:2229] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 LEADER]: Raft consensus shutting down.
I20240502 21:29:28.955638 10981 raft_consensus.cc:2258] T 8839111f2ddd4ae28be77b16966e659f P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.956195 10981 tablet_replica.cc:330] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.956349 10981 raft_consensus.cc:2229] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.956512 10981 raft_consensus.cc:2258] T 085e2f4b603744e1bf1c1394a54dc9bc P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.956952 10981 tablet_replica.cc:330] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.957137 10981 raft_consensus.cc:2229] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.957289 10981 raft_consensus.cc:2258] T 61d83bba66524f9288af49392ab4ed96 P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.957720 10981 tablet_replica.cc:330] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.957870 10981 raft_consensus.cc:2229] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 LEADER]: Raft consensus shutting down.
I20240502 21:29:28.958144 10981 raft_consensus.cc:2258] T 0defef714aa44459a0a1a1a6468dfe0f P fa0b4ce7618d42ff8a424b77f59a89cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.958582 10981 tablet_replica.cc:330] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.958745 10981 raft_consensus.cc:2229] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.958890 10981 raft_consensus.cc:2258] T 0059e9d031fa4b5c9ad9414f80a61529 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.959324 10981 tablet_replica.cc:330] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.959471 10981 raft_consensus.cc:2229] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.959650 10981 raft_consensus.cc:2258] T 9da63340e627438f895d2a7edd31e5c7 P fa0b4ce7618d42ff8a424b77f59a89cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.960125 10981 tablet_replica.cc:330] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.960289 10981 raft_consensus.cc:2229] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 LEADER]: Raft consensus shutting down.
I20240502 21:29:28.960539 10981 raft_consensus.cc:2258] T 272a4fe221da43439bc128eb339ef434 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.960994 10981 tablet_replica.cc:330] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.961136 10981 raft_consensus.cc:2229] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.961297 10981 raft_consensus.cc:2258] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P fa0b4ce7618d42ff8a424b77f59a89cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.961699 10981 tablet_replica.cc:330] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd: stopping tablet replica
I20240502 21:29:28.961855 10981 raft_consensus.cc:2229] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.962010 10981 raft_consensus.cc:2258] T 9a990045515e4479b625327a285c6af3 P fa0b4ce7618d42ff8a424b77f59a89cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.970542 10981 tablet_server.cc:194] TabletServer@127.10.185.65:0 shutdown complete.
W20240502 21:29:28.972635 11718 consensus_peers.cc:485] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a -> Peer fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333): Couldn't send request to peer fa0b4ce7618d42ff8a424b77f59a89cd. Status: Network error: Client connection negotiation failed: client connection to 127.10.185.65:45333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:28.987756 10981 tablet_server.cc:178] TabletServer@127.10.185.66:0 shutting down...
I20240502 21:29:28.996909 10981 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240502 21:29:28.997236 10981 tablet_replica.cc:330] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:28.997450 10981 raft_consensus.cc:2229] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.997648 10981 raft_consensus.cc:2258] T 8839111f2ddd4ae28be77b16966e659f P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.998247 10981 tablet_replica.cc:330] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:28.998402 10981 raft_consensus.cc:2229] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.998548 10981 raft_consensus.cc:2258] T 0defef714aa44459a0a1a1a6468dfe0f P 47a8645e1fc349299fd909d30cf01dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.998996 10981 tablet_replica.cc:330] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:28.999135 10981 raft_consensus.cc:2229] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:28.999260 10981 raft_consensus.cc:2258] T 61d83bba66524f9288af49392ab4ed96 P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:28.999718 10981 tablet_replica.cc:330] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:28.999867 10981 raft_consensus.cc:2229] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.000051 10981 raft_consensus.cc:2258] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 47a8645e1fc349299fd909d30cf01dd1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.000475 10981 tablet_replica.cc:330] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:29.000628 10981 raft_consensus.cc:2229] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 4 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.000888 10981 raft_consensus.cc:2258] T 085e2f4b603744e1bf1c1394a54dc9bc P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.001387 10981 tablet_replica.cc:330] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
W20240502 21:29:29.001444 11718 consensus_peers.cc:485] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a -> Peer fa0b4ce7618d42ff8a424b77f59a89cd (127.10.185.65:45333): Couldn't send request to peer fa0b4ce7618d42ff8a424b77f59a89cd. Status: Network error: Client connection negotiation failed: client connection to 127.10.185.65:45333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:29.001541 10981 raft_consensus.cc:2229] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.001735 10981 raft_consensus.cc:2258] T df366396c6044cf4b642bcc2da2046b3 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.002183 10981 tablet_replica.cc:330] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:29.002318 10981 raft_consensus.cc:2229] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.002449 10981 raft_consensus.cc:2258] T 9da63340e627438f895d2a7edd31e5c7 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.002848 10981 tablet_replica.cc:330] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:29.002975 10981 raft_consensus.cc:2229] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.003240 10981 raft_consensus.cc:2258] T 0059e9d031fa4b5c9ad9414f80a61529 P 47a8645e1fc349299fd909d30cf01dd1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.003636 10981 tablet_replica.cc:330] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:29.003736 10981 raft_consensus.cc:2229] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.004009 10981 raft_consensus.cc:2258] T 9a990045515e4479b625327a285c6af3 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.004416 10981 tablet_replica.cc:330] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1: stopping tablet replica
I20240502 21:29:29.004536 10981 raft_consensus.cc:2229] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.004667 10981 raft_consensus.cc:2258] T 272a4fe221da43439bc128eb339ef434 P 47a8645e1fc349299fd909d30cf01dd1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.021541 10981 tablet_server.cc:194] TabletServer@127.10.185.66:0 shutdown complete.
I20240502 21:29:29.026538 10981 tablet_server.cc:178] TabletServer@127.10.185.67:0 shutting down...
I20240502 21:29:29.036111 10981 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240502 21:29:29.036420 10981 tablet_replica.cc:330] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.036641 10981 raft_consensus.cc:2229] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.036865 10981 raft_consensus.cc:2258] T 9a990045515e4479b625327a285c6af3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.037477 10981 tablet_replica.cc:330] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.037629 10981 raft_consensus.cc:2229] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.037940 10981 raft_consensus.cc:2258] T 9da63340e627438f895d2a7edd31e5c7 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.038412 10981 tablet_replica.cc:330] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.038549 10981 raft_consensus.cc:2229] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.038681 10981 raft_consensus.cc:2258] T 0059e9d031fa4b5c9ad9414f80a61529 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.039091 10981 tablet_replica.cc:330] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.039228 10981 raft_consensus.cc:2229] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.039363 10981 raft_consensus.cc:2258] T 085e2f4b603744e1bf1c1394a54dc9bc P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.039777 10981 tablet_replica.cc:330] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.039918 10981 raft_consensus.cc:2229] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.040099 10981 raft_consensus.cc:2258] T 8839111f2ddd4ae28be77b16966e659f P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.040513 10981 tablet_replica.cc:330] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.040645 10981 raft_consensus.cc:2229] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.040768 10981 raft_consensus.cc:2258] T 272a4fe221da43439bc128eb339ef434 P 152ea2e21dd1407586e9110cfc5e7d5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.041159 10981 tablet_replica.cc:330] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.041290 10981 raft_consensus.cc:2229] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.041566 10981 raft_consensus.cc:2258] T 4b4317ce6a1b451a84ae2ced0fc8ae6d P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.041980 10981 tablet_replica.cc:330] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.042109 10981 raft_consensus.cc:2229] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.042233 10981 raft_consensus.cc:2258] T df366396c6044cf4b642bcc2da2046b3 P 152ea2e21dd1407586e9110cfc5e7d5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.042613 10981 tablet_replica.cc:330] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.042737 10981 raft_consensus.cc:2229] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:29.042877 10981 raft_consensus.cc:2258] T 0defef714aa44459a0a1a1a6468dfe0f P 152ea2e21dd1407586e9110cfc5e7d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.043267 10981 tablet_replica.cc:330] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a: stopping tablet replica
I20240502 21:29:29.043390 10981 raft_consensus.cc:2229] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.043634 10981 raft_consensus.cc:2258] T 61d83bba66524f9288af49392ab4ed96 P 152ea2e21dd1407586e9110cfc5e7d5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.065603 10981 tablet_server.cc:194] TabletServer@127.10.185.67:0 shutdown complete.
I20240502 21:29:29.070667 10981 master.cc:537] Master@127.10.185.126:33339 shutting down...
I20240502 21:29:29.079371 10981 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 1 LEADER]: Raft consensus shutting down.
I20240502 21:29:29.079651 10981 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:29.079759 10981 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3f69d2c980a34635b9467d373bbc0f85: stopping tablet replica
I20240502 21:29:29.096235 10981 master.cc:552] Master@127.10.185.126:33339 shutdown complete.
152ea2e21dd1407586e9110cfc5e7d5a 1
47a8645e1fc349299fd909d30cf01dd1 4
fa0b4ce7618d42ff8a424b77f59a89cd 5
152ea2e21dd1407586e9110cfc5e7d5a 3
47a8645e1fc349299fd909d30cf01dd1 3
fa0b4ce7618d42ff8a424b77f59a89cd 4
152ea2e21dd1407586e9110cfc5e7d5a 1
47a8645e1fc349299fd909d30cf01dd1 1
fa0b4ce7618d42ff8a424b77f59a89cd 8
152ea2e21dd1407586e9110cfc5e7d5a 3
47a8645e1fc349299fd909d30cf01dd1 3
fa0b4ce7618d42ff8a424b77f59a89cd 4
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (27485 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20240502 21:29:29.114537 10981 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.185.126:42659
I20240502 21:29:29.115100 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:29.118095 11971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:29.118986 11972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:29.119716 11974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:29.120285 10981 server_base.cc:1034] running on GCE node
I20240502 21:29:29.120623 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:29.120693 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:29.120745 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685369120740 us; error 0 us; skew 500 ppm
I20240502 21:29:29.120965 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:29.123390 10981 webserver.cc:460] Webserver started at http://127.10.185.126:37103/ using document root <none> and password file <none>
I20240502 21:29:29.123641 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:29.123713 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:29.123847 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:29.124594 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/master-0-root/instance:
uuid: "e79f9a2e4b9f42228d08cbd61fd01028"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.127394 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:29.129443 11979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.129876 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20240502 21:29:29.129985 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/master-0-root
uuid: "e79f9a2e4b9f42228d08cbd61fd01028"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.130095 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:29.164634 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:29.165230 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:29.178252 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.126:42659
I20240502 21:29:29.178275 12031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.126:42659 every 8 connection(s)
I20240502 21:29:29.179692 12032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.184113 12032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028: Bootstrap starting.
I20240502 21:29:29.185549 12032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.187305 12032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028: No bootstrap required, opened a new log
I20240502 21:29:29.187942 12032 raft_consensus.cc:348] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" member_type: VOTER }
I20240502 21:29:29.188104 12032 raft_consensus.cc:374] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.188154 12032 raft_consensus.cc:729] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79f9a2e4b9f42228d08cbd61fd01028, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.188354 12032 consensus_queue.cc:260] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" member_type: VOTER }
I20240502 21:29:29.188477 12032 raft_consensus.cc:388] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 21:29:29.188540 12032 raft_consensus.cc:482] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 21:29:29.188637 12032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:29.190471 12032 raft_consensus.cc:504] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" member_type: VOTER }
I20240502 21:29:29.190678 12032 leader_election.cc:302] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e79f9a2e4b9f42228d08cbd61fd01028; no voters:
I20240502 21:29:29.191095 12032 leader_election.cc:288] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 21:29:29.191231 12036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:29.191730 12036 raft_consensus.cc:686] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [term 1 LEADER]: Becoming Leader. State: Replica: e79f9a2e4b9f42228d08cbd61fd01028, State: Running, Role: LEADER
I20240502 21:29:29.192009 12032 sys_catalog.cc:564] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 21:29:29.192023 12036 consensus_queue.cc:237] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" member_type: VOTER }
I20240502 21:29:29.193249 12037 sys_catalog.cc:455] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" member_type: VOTER } }
I20240502 21:29:29.193270 12038 sys_catalog.cc:455] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e79f9a2e4b9f42228d08cbd61fd01028. Latest consensus state: current_term: 1 leader_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e79f9a2e4b9f42228d08cbd61fd01028" member_type: VOTER } }
I20240502 21:29:29.193588 12037 sys_catalog.cc:458] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [sys.catalog]: This master's current role is: LEADER
I20240502 21:29:29.193610 12038 sys_catalog.cc:458] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028 [sys.catalog]: This master's current role is: LEADER
I20240502 21:29:29.195021 12042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 21:29:29.197151 12042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 21:29:29.197674 10981 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240502 21:29:29.200274 12042 catalog_manager.cc:1347] Generated new cluster ID: 1d861d978f6a44ba8161df65bbf639ee
I20240502 21:29:29.200350 12042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 21:29:29.213716 12042 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 21:29:29.214520 12042 catalog_manager.cc:1504] Loading token signing keys...
I20240502 21:29:29.223637 12042 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e79f9a2e4b9f42228d08cbd61fd01028: Generated new TSK 0
I20240502 21:29:29.223886 12042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 21:29:29.230546 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:29.234098 12054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:29.235064 12055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:29.235808 10981 server_base.cc:1034] running on GCE node
W20240502 21:29:29.236308 12057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:29.236704 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:29.236768 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:29.236799 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685369236795 us; error 0 us; skew 500 ppm
I20240502 21:29:29.236922 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:29.238296 10981 webserver.cc:460] Webserver started at http://127.10.185.65:41535/ using document root <none> and password file <none>
I20240502 21:29:29.238526 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:29.238586 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:29.238679 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:29.239323 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root/instance:
uuid: "0672163bb07b41a1b14342a8efcb24a0"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.242077 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20240502 21:29:29.243732 12062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.244158 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20240502 21:29:29.244277 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root
uuid: "0672163bb07b41a1b14342a8efcb24a0"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.244393 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:29.260797 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:29.261431 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:29.262058 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:29.262966 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:29.263025 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.263125 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:29.263164 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.278770 12126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.65:44241 every 8 connection(s)
I20240502 21:29:29.278770 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.65:44241
I20240502 21:29:29.280568 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:29.284204 12131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:29.284842 12132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:29.285610 12134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:29.286022 10981 server_base.cc:1034] running on GCE node
I20240502 21:29:29.286545 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:29.286626 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:29.286653 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685369286650 us; error 0 us; skew 500 ppm
I20240502 21:29:29.286828 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:29.287933 12127 heartbeater.cc:347] Connected to a master server at 127.10.185.126:42659
I20240502 21:29:29.288095 12127 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:29.288479 12127 heartbeater.cc:511] Master 127.10.185.126:42659 requested a full tablet report, sending...
I20240502 21:29:29.288913 10981 webserver.cc:460] Webserver started at http://127.10.185.66:38231/ using document root <none> and password file <none>
I20240502 21:29:29.289184 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:29.289254 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:29.289391 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:29.289499 11996 ts_manager.cc:194] Registered new tserver with Master: 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241)
I20240502 21:29:29.290019 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root/instance:
uuid: "17f04379252d4ecfab3a4916c61a984f"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.290560 11996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42778
I20240502 21:29:29.292793 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20240502 21:29:29.294462 12139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.294836 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20240502 21:29:29.294934 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root
uuid: "17f04379252d4ecfab3a4916c61a984f"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.295050 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:29.315186 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:29.315817 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:29.316479 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:29.317345 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:29.317414 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.317484 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:29.317531 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.331311 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.66:34017
I20240502 21:29:29.331341 12203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.66:34017 every 8 connection(s)
I20240502 21:29:29.332989 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:29.336633 12207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:29.337466 12208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:29.337857 10981 server_base.cc:1034] running on GCE node
W20240502 21:29:29.338423 12210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:29.338968 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:29.339038 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:29.339085 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685369339080 us; error 0 us; skew 500 ppm
I20240502 21:29:29.339228 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:29.340494 12204 heartbeater.cc:347] Connected to a master server at 127.10.185.126:42659
I20240502 21:29:29.340620 12204 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:29.340893 10981 webserver.cc:460] Webserver started at http://127.10.185.67:38457/ using document root <none> and password file <none>
I20240502 21:29:29.341065 12204 heartbeater.cc:511] Master 127.10.185.126:42659 requested a full tablet report, sending...
I20240502 21:29:29.341116 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:29.341189 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:29.341303 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:29.341967 11996 ts_manager.cc:194] Registered new tserver with Master: 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:29.342046 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root/instance:
uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.342656 11996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42792
I20240502 21:29:29.344871 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 21:29:29.346521 12215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.346846 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240502 21:29:29.346962 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root
uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
format_stamp: "Formatted at 2024-05-02 21:29:29 on dist-test-slave-ccs3"
I20240502 21:29:29.347071 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:29.359639 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:29.360354 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:29.360961 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:29.361835 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:29.361894 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.361976 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:29.362016 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:29.375722 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.67:34725
I20240502 21:29:29.375766 12278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.67:34725 every 8 connection(s)
I20240502 21:29:29.382867 12279 heartbeater.cc:347] Connected to a master server at 127.10.185.126:42659
I20240502 21:29:29.383014 12279 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:29.383363 12279 heartbeater.cc:511] Master 127.10.185.126:42659 requested a full tablet report, sending...
I20240502 21:29:29.384328 11996 ts_manager.cc:194] Registered new tserver with Master: 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:29.384630 10981 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007954331s
I20240502 21:29:29.385149 11996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42808
I20240502 21:29:29.385654 10981 test_util.cc:268] Using random seed: 1280836274
I20240502 21:29:29.398340 11996 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42816:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240502 21:29:29.399832 11996 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240502 21:29:29.451185 12090 tablet_service.cc:1467] Processing CreateTablet for tablet 6d05591283c24d0d9e6de5ad16ab504d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240502 21:29:29.452013 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d05591283c24d0d9e6de5ad16ab504d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.454059 12089 tablet_service.cc:1467] Processing CreateTablet for tablet dafaa80ca77c464e948c967cc0a32055 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240502 21:29:29.454685 12089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafaa80ca77c464e948c967cc0a32055. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.464819 12088 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3aa4f038da4e6596a28265565b2023 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240502 21:29:29.465132 12084 tablet_service.cc:1467] Processing CreateTablet for tablet c9758cbdd5b845d6883c6e34d0e3f6a6 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240502 21:29:29.465426 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3aa4f038da4e6596a28265565b2023. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.465739 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9758cbdd5b845d6883c6e34d0e3f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.466153 12087 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0b9621b33c4afcb3842c10ac4c7148 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240502 21:29:29.466588 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0b9621b33c4afcb3842c10ac4c7148. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.472456 12086 tablet_service.cc:1467] Processing CreateTablet for tablet b5847bedd7574bd28b9eeeca3f3cf6f9 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240502 21:29:29.473001 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5847bedd7574bd28b9eeeca3f3cf6f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.476070 12084 tablet_service.cc:1467] Processing CreateTablet for tablet c069ca9b95cc4e8ea75c71ea6cefdcc3 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240502 21:29:29.476310 12085 tablet_service.cc:1467] Processing CreateTablet for tablet b56cd8f7bdee468b846b4f51e86e1f27 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240502 21:29:29.476693 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c069ca9b95cc4e8ea75c71ea6cefdcc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.476972 12083 tablet_service.cc:1467] Processing CreateTablet for tablet 77279927ddf545e9a3216606b9bab54d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240502 21:29:29.477380 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77279927ddf545e9a3216606b9bab54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.480327 12082 tablet_service.cc:1467] Processing CreateTablet for tablet b6d88ae6514545bca16f20beb52b8e7b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240502 21:29:29.480790 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d88ae6514545bca16f20beb52b8e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.481426 12089 tablet_service.cc:1467] Processing CreateTablet for tablet 59bec182656a4200b4e9938e861ab21f (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240502 21:29:29.482479 12089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59bec182656a4200b4e9938e861ab21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.484331 12081 tablet_service.cc:1467] Processing CreateTablet for tablet da050ef2be604dd5af90e13e6d70c74d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240502 21:29:29.484788 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da050ef2be604dd5af90e13e6d70c74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.488114 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56cd8f7bdee468b846b4f51e86e1f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.494944 12084 tablet_service.cc:1467] Processing CreateTablet for tablet fe982f82c003411e9f9dda42378399f8 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240502 21:29:29.495510 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe982f82c003411e9f9dda42378399f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.498519 12166 tablet_service.cc:1467] Processing CreateTablet for tablet dafaa80ca77c464e948c967cc0a32055 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240502 21:29:29.499053 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafaa80ca77c464e948c967cc0a32055. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.500298 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 6d05591283c24d0d9e6de5ad16ab504d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240502 21:29:29.500726 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d05591283c24d0d9e6de5ad16ab504d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.504351 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 59bec182656a4200b4e9938e861ab21f (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240502 21:29:29.504670 12162 tablet_service.cc:1467] Processing CreateTablet for tablet b5847bedd7574bd28b9eeeca3f3cf6f9 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240502 21:29:29.504359 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3aa4f038da4e6596a28265565b2023 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240502 21:29:29.505864 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59bec182656a4200b4e9938e861ab21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.505885 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0b9621b33c4afcb3842c10ac4c7148 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240502 21:29:29.506281 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0b9621b33c4afcb3842c10ac4c7148. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.509596 12159 tablet_service.cc:1467] Processing CreateTablet for tablet 77279927ddf545e9a3216606b9bab54d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240502 21:29:29.510154 12161 tablet_service.cc:1467] Processing CreateTablet for tablet b56cd8f7bdee468b846b4f51e86e1f27 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240502 21:29:29.510257 12087 tablet_service.cc:1467] Processing CreateTablet for tablet a82c29bb30eb4515ae709c0144a8998b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240502 21:29:29.510607 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82c29bb30eb4515ae709c0144a8998b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.509680 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5847bedd7574bd28b9eeeca3f3cf6f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.511215 12083 tablet_service.cc:1467] Processing CreateTablet for tablet 434ffa8e08cf45f6905eb6aaaab820ca (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240502 21:29:29.511554 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434ffa8e08cf45f6905eb6aaaab820ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.511811 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77279927ddf545e9a3216606b9bab54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.510509 12158 tablet_service.cc:1467] Processing CreateTablet for tablet b6d88ae6514545bca16f20beb52b8e7b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240502 21:29:29.512327 12160 tablet_service.cc:1467] Processing CreateTablet for tablet c9758cbdd5b845d6883c6e34d0e3f6a6 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240502 21:29:29.513353 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3aa4f038da4e6596a28265565b2023. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.513581 12160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9758cbdd5b845d6883c6e34d0e3f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.513818 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56cd8f7bdee468b846b4f51e86e1f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.518779 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d88ae6514545bca16f20beb52b8e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.524699 12084 tablet_service.cc:1467] Processing CreateTablet for tablet b0766fbf50cb4b17a55ee3a21de8248c (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240502 21:29:29.525203 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0766fbf50cb4b17a55ee3a21de8248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.528990 12090 tablet_service.cc:1467] Processing CreateTablet for tablet 2924f688055f461ba9c7c0e70197376a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240502 21:29:29.529404 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2924f688055f461ba9c7c0e70197376a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.532586 12236 tablet_service.cc:1467] Processing CreateTablet for tablet c9758cbdd5b845d6883c6e34d0e3f6a6 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240502 21:29:29.533140 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9758cbdd5b845d6883c6e34d0e3f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.535720 12085 tablet_service.cc:1467] Processing CreateTablet for tablet 09266cc9ff6c453bbf4cd0a1621375db (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240502 21:29:29.536201 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09266cc9ff6c453bbf4cd0a1621375db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.536026 12238 tablet_service.cc:1467] Processing CreateTablet for tablet b5847bedd7574bd28b9eeeca3f3cf6f9 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240502 21:29:29.538810 12243 tablet_service.cc:1467] Processing CreateTablet for tablet 6d05591283c24d0d9e6de5ad16ab504d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240502 21:29:29.538941 12235 tablet_service.cc:1467] Processing CreateTablet for tablet 77279927ddf545e9a3216606b9bab54d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240502 21:29:29.539680 12239 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0b9621b33c4afcb3842c10ac4c7148 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240502 21:29:29.540426 12240 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3aa4f038da4e6596a28265565b2023 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240502 21:29:29.540541 12241 tablet_service.cc:1467] Processing CreateTablet for tablet 59bec182656a4200b4e9938e861ab21f (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240502 21:29:29.541399 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5847bedd7574bd28b9eeeca3f3cf6f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.544157 12237 tablet_service.cc:1467] Processing CreateTablet for tablet b56cd8f7bdee468b846b4f51e86e1f27 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240502 21:29:29.546147 12242 tablet_service.cc:1467] Processing CreateTablet for tablet dafaa80ca77c464e948c967cc0a32055 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240502 21:29:29.547498 12294 tablet_bootstrap.cc:492] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:29.547863 12243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d05591283c24d0d9e6de5ad16ab504d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.548115 12234 tablet_service.cc:1467] Processing CreateTablet for tablet b6d88ae6514545bca16f20beb52b8e7b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240502 21:29:29.550645 12241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59bec182656a4200b4e9938e861ab21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.550889 12239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0b9621b33c4afcb3842c10ac4c7148. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.557122 12082 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6b3c759a8b4236b3712d5161bdb8dc (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240502 21:29:29.557546 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6b3c759a8b4236b3712d5161bdb8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.558869 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77279927ddf545e9a3216606b9bab54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.561198 12161 tablet_service.cc:1467] Processing CreateTablet for tablet c069ca9b95cc4e8ea75c71ea6cefdcc3 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240502 21:29:29.561543 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c069ca9b95cc4e8ea75c71ea6cefdcc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.563832 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d88ae6514545bca16f20beb52b8e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.566732 12166 tablet_service.cc:1467] Processing CreateTablet for tablet da050ef2be604dd5af90e13e6d70c74d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240502 21:29:29.567190 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da050ef2be604dd5af90e13e6d70c74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.570924 12242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafaa80ca77c464e948c967cc0a32055. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.571096 12167 tablet_service.cc:1467] Processing CreateTablet for tablet fe982f82c003411e9f9dda42378399f8 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240502 21:29:29.571353 12240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3aa4f038da4e6596a28265565b2023. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.571534 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe982f82c003411e9f9dda42378399f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.572388 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56cd8f7bdee468b846b4f51e86e1f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.573057 12081 tablet_service.cc:1467] Processing CreateTablet for tablet e288a51b369a41ffa0a8d2dedb849c24 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240502 21:29:29.573416 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e288a51b369a41ffa0a8d2dedb849c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.576819 12083 tablet_service.cc:1467] Processing CreateTablet for tablet 244c153efb7943f48ddee849bb0187c4 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240502 21:29:29.577698 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244c153efb7943f48ddee849bb0187c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.583004 12088 tablet_service.cc:1467] Processing CreateTablet for tablet a1a147afb591461387bfe6ed7587a6ba (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240502 21:29:29.583384 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 434ffa8e08cf45f6905eb6aaaab820ca (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240502 21:29:29.583384 12086 tablet_service.cc:1467] Processing CreateTablet for tablet d6c0955962874018b574931aea7bd0ce (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240502 21:29:29.583839 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434ffa8e08cf45f6905eb6aaaab820ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.583479 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a147afb591461387bfe6ed7587a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.585251 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c0955962874018b574931aea7bd0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.585238 12089 tablet_service.cc:1467] Processing CreateTablet for tablet a4afa7764f554f62abb539fe13838764 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240502 21:29:29.585642 12089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4afa7764f554f62abb539fe13838764. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.585896 12295 tablet_bootstrap.cc:492] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:29.588104 12294 tablet_bootstrap.cc:654] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.598266 12164 tablet_service.cc:1467] Processing CreateTablet for tablet a82c29bb30eb4515ae709c0144a8998b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240502 21:29:29.601013 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82c29bb30eb4515ae709c0144a8998b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.617817 12162 tablet_service.cc:1467] Processing CreateTablet for tablet b0766fbf50cb4b17a55ee3a21de8248c (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240502 21:29:29.622354 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0766fbf50cb4b17a55ee3a21de8248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.626268 12241 tablet_service.cc:1467] Processing CreateTablet for tablet da050ef2be604dd5af90e13e6d70c74d (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240502 21:29:29.634922 12241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da050ef2be604dd5af90e13e6d70c74d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.644737 12243 tablet_service.cc:1467] Processing CreateTablet for tablet c069ca9b95cc4e8ea75c71ea6cefdcc3 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240502 21:29:29.645407 12243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c069ca9b95cc4e8ea75c71ea6cefdcc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.649250 12238 tablet_service.cc:1467] Processing CreateTablet for tablet fe982f82c003411e9f9dda42378399f8 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240502 21:29:29.650528 12160 tablet_service.cc:1467] Processing CreateTablet for tablet 2924f688055f461ba9c7c0e70197376a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240502 21:29:29.657603 12090 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9bd1fc947f428197d8747df293dffd (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240502 21:29:29.658689 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9bd1fc947f428197d8747df293dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.658852 12160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2924f688055f461ba9c7c0e70197376a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.659464 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe982f82c003411e9f9dda42378399f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.660068 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 09266cc9ff6c453bbf4cd0a1621375db (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240502 21:29:29.660670 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09266cc9ff6c453bbf4cd0a1621375db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.662416 12159 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6b3c759a8b4236b3712d5161bdb8dc (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240502 21:29:29.662905 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6b3c759a8b4236b3712d5161bdb8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.669860 12295 tablet_bootstrap.cc:654] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.672446 12158 tablet_service.cc:1467] Processing CreateTablet for tablet e288a51b369a41ffa0a8d2dedb849c24 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240502 21:29:29.675535 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e288a51b369a41ffa0a8d2dedb849c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.683842 12294 tablet_bootstrap.cc:492] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:29.684067 12294 ts_tablet_manager.cc:1367] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.137s user 0.021s sys 0.047s
I20240502 21:29:29.684899 12294 raft_consensus.cc:348] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.685078 12294 raft_consensus.cc:374] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.685114 12294 raft_consensus.cc:729] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.686626 12236 tablet_service.cc:1467] Processing CreateTablet for tablet 434ffa8e08cf45f6905eb6aaaab820ca (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240502 21:29:29.687655 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434ffa8e08cf45f6905eb6aaaab820ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.688162 12242 tablet_service.cc:1467] Processing CreateTablet for tablet a82c29bb30eb4515ae709c0144a8998b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240502 21:29:29.688673 12242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82c29bb30eb4515ae709c0144a8998b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.685312 12294 consensus_queue.cc:260] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.692062 12297 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:29.692348 12085 tablet_service.cc:1467] Processing CreateTablet for tablet aa48c97f4426444585651368f28b265a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240502 21:29:29.693102 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa48c97f4426444585651368f28b265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.703306 12081 tablet_service.cc:1467] Processing CreateTablet for tablet db39150ee1094ace88cbc8a2c105985a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240502 21:29:29.703775 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db39150ee1094ace88cbc8a2c105985a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.705127 12237 tablet_service.cc:1467] Processing CreateTablet for tablet b0766fbf50cb4b17a55ee3a21de8248c (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240502 21:29:29.710529 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0766fbf50cb4b17a55ee3a21de8248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.726591 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 244c153efb7943f48ddee849bb0187c4 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240502 21:29:29.726590 12087 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5d3606b31841f6986c1c0954eb9332 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240502 21:29:29.729053 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244c153efb7943f48ddee849bb0187c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.729799 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5d3606b31841f6986c1c0954eb9332. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.733465 12239 tablet_service.cc:1467] Processing CreateTablet for tablet 2924f688055f461ba9c7c0e70197376a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240502 21:29:29.734648 12294 ts_tablet_manager.cc:1398] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.050s user 0.024s sys 0.016s
I20240502 21:29:29.734959 12127 heartbeater.cc:503] Master 127.10.185.126:42659 was elected leader, sending a full tablet report...
I20240502 21:29:29.735028 12294 tablet_bootstrap.cc:492] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:29.736367 12164 tablet_service.cc:1467] Processing CreateTablet for tablet a4afa7764f554f62abb539fe13838764 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240502 21:29:29.736832 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4afa7764f554f62abb539fe13838764. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.736995 12294 tablet_bootstrap.cc:654] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.739565 12295 tablet_bootstrap.cc:492] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:29.739725 12295 ts_tablet_manager.cc:1367] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.159s user 0.004s sys 0.005s
I20240502 21:29:29.741144 12084 tablet_service.cc:1467] Processing CreateTablet for tablet 6aef08a96a4e49f6bc1d3fc3175d1685 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240502 21:29:29.741711 12239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2924f688055f461ba9c7c0e70197376a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.742233 12241 tablet_service.cc:1467] Processing CreateTablet for tablet 09266cc9ff6c453bbf4cd0a1621375db (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240502 21:29:29.743665 12241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09266cc9ff6c453bbf4cd0a1621375db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.746387 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aef08a96a4e49f6bc1d3fc3175d1685. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.746615 12235 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6b3c759a8b4236b3712d5161bdb8dc (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240502 21:29:29.747699 12086 tablet_service.cc:1467] Processing CreateTablet for tablet 0c958a20984244369aeed6a8442d51dd (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240502 21:29:29.750838 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6b3c759a8b4236b3712d5161bdb8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.751420 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c958a20984244369aeed6a8442d51dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.753746 12234 tablet_service.cc:1467] Processing CreateTablet for tablet e288a51b369a41ffa0a8d2dedb849c24 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240502 21:29:29.753295 12295 raft_consensus.cc:348] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.754189 12295 raft_consensus.cc:374] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.754263 12295 raft_consensus.cc:729] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.761301 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e288a51b369a41ffa0a8d2dedb849c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.754534 12295 consensus_queue.cc:260] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.763619 12165 tablet_service.cc:1467] Processing CreateTablet for tablet a1a147afb591461387bfe6ed7587a6ba (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240502 21:29:29.763999 12082 tablet_service.cc:1467] Processing CreateTablet for tablet 494e0f94e6b94037b90b833582f3670f (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240502 21:29:29.765929 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a147afb591461387bfe6ed7587a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.766721 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494e0f94e6b94037b90b833582f3670f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.767830 12161 tablet_service.cc:1467] Processing CreateTablet for tablet d6c0955962874018b574931aea7bd0ce (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240502 21:29:29.768286 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c0955962874018b574931aea7bd0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.769783 12088 tablet_service.cc:1467] Processing CreateTablet for tablet 17e15a0d7054443ea55550d20ee0518e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240502 21:29:29.770159 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e15a0d7054443ea55550d20ee0518e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.774546 12294 tablet_bootstrap.cc:492] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:29.774745 12294 ts_tablet_manager.cc:1367] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
W20240502 21:29:29.780248 12128 tablet.cc:2369] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 21:29:29.780515 12297 tablet_bootstrap.cc:654] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.783284 12085 tablet_service.cc:1467] Processing CreateTablet for tablet 834684d85d644236aab5229b549c428a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240502 21:29:29.783485 12243 tablet_service.cc:1467] Processing CreateTablet for tablet 244c153efb7943f48ddee849bb0187c4 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240502 21:29:29.784099 12243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244c153efb7943f48ddee849bb0187c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.787009 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834684d85d644236aab5229b549c428a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.788219 12240 tablet_service.cc:1467] Processing CreateTablet for tablet a4afa7764f554f62abb539fe13838764 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240502 21:29:29.788671 12240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4afa7764f554f62abb539fe13838764. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.789453 12083 tablet_service.cc:1467] Processing CreateTablet for tablet 81abff8d58dc4c69b08ac2a161748b5e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240502 21:29:29.789840 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81abff8d58dc4c69b08ac2a161748b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.792740 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9bd1fc947f428197d8747df293dffd (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240502 21:29:29.796029 12294 raft_consensus.cc:348] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.796252 12294 raft_consensus.cc:374] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.796162 12158 tablet_service.cc:1467] Processing CreateTablet for tablet aa48c97f4426444585651368f28b265a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240502 21:29:29.796321 12294 raft_consensus.cc:729] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.796581 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa48c97f4426444585651368f28b265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.796660 12166 tablet_service.cc:1467] Processing CreateTablet for tablet db39150ee1094ace88cbc8a2c105985a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240502 21:29:29.798287 12159 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5d3606b31841f6986c1c0954eb9332 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240502 21:29:29.796748 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9bd1fc947f428197d8747df293dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.799549 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5d3606b31841f6986c1c0954eb9332. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.799484 12294 consensus_queue.cc:260] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.800009 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db39150ee1094ace88cbc8a2c105985a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.800089 12294 ts_tablet_manager.cc:1398] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.023s user 0.002s sys 0.000s
I20240502 21:29:29.800356 12294 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:29.801627 12294 tablet_bootstrap.cc:654] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.803058 12081 tablet_service.cc:1467] Processing CreateTablet for tablet ad85ac6e9a824608ad426e1453c70237 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240502 21:29:29.803462 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad85ac6e9a824608ad426e1453c70237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.803711 12294 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:29.803910 12294 ts_tablet_manager.cc:1367] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:29.804790 12294 raft_consensus.cc:348] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.805058 12294 raft_consensus.cc:374] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.805168 12294 raft_consensus.cc:729] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.805478 12294 consensus_queue.cc:260] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.806635 12295 ts_tablet_manager.cc:1398] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.067s user 0.015s sys 0.010s
I20240502 21:29:29.806711 12166 tablet_service.cc:1467] Processing CreateTablet for tablet 6aef08a96a4e49f6bc1d3fc3175d1685 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240502 21:29:29.806901 12204 heartbeater.cc:503] Master 127.10.185.126:42659 was elected leader, sending a full tablet report...
I20240502 21:29:29.806973 12295 tablet_bootstrap.cc:492] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:29.808910 12234 tablet_service.cc:1467] Processing CreateTablet for tablet a1a147afb591461387bfe6ed7587a6ba (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240502 21:29:29.809314 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a147afb591461387bfe6ed7587a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.811012 12238 tablet_service.cc:1467] Processing CreateTablet for tablet d6c0955962874018b574931aea7bd0ce (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240502 21:29:29.811367 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c0955962874018b574931aea7bd0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.813145 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 0c958a20984244369aeed6a8442d51dd (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240502 21:29:29.813221 12242 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9bd1fc947f428197d8747df293dffd (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240502 21:29:29.813558 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c958a20984244369aeed6a8442d51dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.813602 12242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9bd1fc947f428197d8747df293dffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.818359 12298 raft_consensus.cc:482] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:29.818491 12294 ts_tablet_manager.cc:1398] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20240502 21:29:29.818521 12298 raft_consensus.cc:504] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.818758 12294 tablet_bootstrap.cc:492] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:29.820482 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aef08a96a4e49f6bc1d3fc3175d1685. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.822526 12089 tablet_service.cc:1467] Processing CreateTablet for tablet 9b8b568beaec444faa5e779e0b2059b1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240502 21:29:29.822973 12089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b8b568beaec444faa5e779e0b2059b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.824542 12167 tablet_service.cc:1467] Processing CreateTablet for tablet 494e0f94e6b94037b90b833582f3670f (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240502 21:29:29.829289 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494e0f94e6b94037b90b833582f3670f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.830648 12090 tablet_service.cc:1467] Processing CreateTablet for tablet d4fbdb33d94f4dd196663efa7f5b1cd7 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240502 21:29:29.831038 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fbdb33d94f4dd196663efa7f5b1cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.834908 12298 leader_election.cc:288] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:29.836702 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 17e15a0d7054443ea55550d20ee0518e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240502 21:29:29.837141 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e15a0d7054443ea55550d20ee0518e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.841059 12294 tablet_bootstrap.cc:654] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.844944 12234 tablet_service.cc:1467] Processing CreateTablet for tablet aa48c97f4426444585651368f28b265a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240502 21:29:29.845005 12081 tablet_service.cc:1467] Processing CreateTablet for tablet e8c09277218147c6a65b7d984a13c9b6 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240502 21:29:29.845223 12084 tablet_service.cc:1467] Processing CreateTablet for tablet af9adc5b5ce04374bf15c9088b4397d8 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240502 21:29:29.845569 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9adc5b5ce04374bf15c9088b4397d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.846123 12295 tablet_bootstrap.cc:654] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:29.846190 12160 tablet_service.cc:1467] Processing CreateTablet for tablet 834684d85d644236aab5229b549c428a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240502 21:29:29.845005 12237 tablet_service.cc:1467] Processing CreateTablet for tablet db39150ee1094ace88cbc8a2c105985a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240502 21:29:29.847041 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db39150ee1094ace88cbc8a2c105985a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.850392 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c09277218147c6a65b7d984a13c9b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.852172 12160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834684d85d644236aab5229b549c428a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.854820 12240 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5d3606b31841f6986c1c0954eb9332 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240502 21:29:29.855978 12240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5d3606b31841f6986c1c0954eb9332. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.855317 12236 tablet_service.cc:1467] Processing CreateTablet for tablet 6aef08a96a4e49f6bc1d3fc3175d1685 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240502 21:29:29.856539 12241 tablet_service.cc:1467] Processing CreateTablet for tablet 0c958a20984244369aeed6a8442d51dd (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240502 21:29:29.857088 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aef08a96a4e49f6bc1d3fc3175d1685. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.858606 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa48c97f4426444585651368f28b265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.858886 12241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c958a20984244369aeed6a8442d51dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.880678 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 81abff8d58dc4c69b08ac2a161748b5e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240502 21:29:29.882176 12235 tablet_service.cc:1467] Processing CreateTablet for tablet 494e0f94e6b94037b90b833582f3670f (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240502 21:29:29.882787 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494e0f94e6b94037b90b833582f3670f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.884618 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81abff8d58dc4c69b08ac2a161748b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.903484 12085 tablet_service.cc:1467] Processing CreateTablet for tablet 04a7f75fa74945c09343353c01f4b31b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240502 21:29:29.904265 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a7f75fa74945c09343353c01f4b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.911020 12159 tablet_service.cc:1467] Processing CreateTablet for tablet ad85ac6e9a824608ad426e1453c70237 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240502 21:29:29.912278 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad85ac6e9a824608ad426e1453c70237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.916332 12243 tablet_service.cc:1467] Processing CreateTablet for tablet 17e15a0d7054443ea55550d20ee0518e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240502 21:29:29.917366 12295 tablet_bootstrap.cc:492] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:29.917275 12158 tablet_service.cc:1467] Processing CreateTablet for tablet 9b8b568beaec444faa5e779e0b2059b1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240502 21:29:29.917533 12295 ts_tablet_manager.cc:1367] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.111s user 0.007s sys 0.012s
I20240502 21:29:29.917927 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b8b568beaec444faa5e779e0b2059b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.917999 12243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e15a0d7054443ea55550d20ee0518e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.922255 12238 tablet_service.cc:1467] Processing CreateTablet for tablet 834684d85d644236aab5229b549c428a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240502 21:29:29.923825 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834684d85d644236aab5229b549c428a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.928329 12294 tablet_bootstrap.cc:492] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:29.928534 12294 ts_tablet_manager.cc:1367] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.110s user 0.005s sys 0.007s
I20240502 21:29:29.929076 12242 tablet_service.cc:1467] Processing CreateTablet for tablet 81abff8d58dc4c69b08ac2a161748b5e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240502 21:29:29.930835 12242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81abff8d58dc4c69b08ac2a161748b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.933205 12294 raft_consensus.cc:348] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.933382 12294 raft_consensus.cc:374] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.933421 12294 raft_consensus.cc:729] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.934207 12294 consensus_queue.cc:260] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.938660 12294 ts_tablet_manager.cc:1398] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20240502 21:29:29.939098 12294 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:29.936511 12295 raft_consensus.cc:348] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.940480 12295 raft_consensus.cc:374] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.940568 12295 raft_consensus.cc:729] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.942274 12298 raft_consensus.cc:482] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:29.942401 12298 raft_consensus.cc:504] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.942821 12166 tablet_service.cc:1467] Processing CreateTablet for tablet d4fbdb33d94f4dd196663efa7f5b1cd7 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240502 21:29:29.943342 12295 consensus_queue.cc:260] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.944578 12163 tablet_service.cc:1467] Processing CreateTablet for tablet af9adc5b5ce04374bf15c9088b4397d8 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240502 21:29:29.946277 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9adc5b5ce04374bf15c9088b4397d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.948493 12298 leader_election.cc:288] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:29.949160 12083 tablet_service.cc:1467] Processing CreateTablet for tablet 73d4e5a4db934406b9f995442ec6bcd1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240502 21:29:29.950266 12239 tablet_service.cc:1467] Processing CreateTablet for tablet ad85ac6e9a824608ad426e1453c70237 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240502 21:29:29.950636 12239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad85ac6e9a824608ad426e1453c70237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.953837 12297 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:29.954008 12297 ts_tablet_manager.cc:1367] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.264s user 0.012s sys 0.017s
I20240502 21:29:29.954639 12295 ts_tablet_manager.cc:1398] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.037s user 0.001s sys 0.001s
I20240502 21:29:29.954933 12295 tablet_bootstrap.cc:492] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:29.958065 12297 raft_consensus.cc:348] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.958313 12297 raft_consensus.cc:374] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:29.958379 12297 raft_consensus.cc:729] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:29.958617 12297 consensus_queue.cc:260] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:29.961385 12086 tablet_service.cc:1467] Processing CreateTablet for tablet 060e5a3ff13f40629080ec7036eb39ef (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240502 21:29:29.961882 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e5a3ff13f40629080ec7036eb39ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.964226 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fbdb33d94f4dd196663efa7f5b1cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.971619 12237 tablet_service.cc:1467] Processing CreateTablet for tablet 9b8b568beaec444faa5e779e0b2059b1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240502 21:29:29.972082 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b8b568beaec444faa5e779e0b2059b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.973030 12087 tablet_service.cc:1467] Processing CreateTablet for tablet 528a557fdd90485d91077d2a61095878 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240502 21:29:29.973381 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528a557fdd90485d91077d2a61095878. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.977669 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d4e5a4db934406b9f995442ec6bcd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.968719 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafaa80ca77c464e948c967cc0a32055" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:29.979990 12177 raft_consensus.cc:2454] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:29.980440 12066 leader_election.cc:302] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:29.980792 12298 raft_consensus.cc:2781] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:29.980860 12298 raft_consensus.cc:482] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:29.980901 12298 raft_consensus.cc:3037] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:29.968860 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9758cbdd5b845d6883c6e34d0e3f6a6" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:29.988157 12167 tablet_service.cc:1467] Processing CreateTablet for tablet e8c09277218147c6a65b7d984a13c9b6 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240502 21:29:29.988883 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c09277218147c6a65b7d984a13c9b6. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 21:29:29.989326 12066 leader_election.cc:341] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:29.985407 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafaa80ca77c464e948c967cc0a32055" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:29.995340 12088 tablet_service.cc:1467] Processing CreateTablet for tablet 1b8307fd623a499fa05cd9eca238ddc2 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240502 21:29:29.995760 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8307fd623a499fa05cd9eca238ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:29.996472 12297 ts_tablet_manager.cc:1398] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.042s user 0.008s sys 0.028s
I20240502 21:29:29.996742 12279 heartbeater.cc:503] Master 127.10.185.126:42659 was elected leader, sending a full tablet report...
W20240502 21:29:30.000181 12065 leader_election.cc:341] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:29.985610 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9758cbdd5b845d6883c6e34d0e3f6a6" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.002632 12252 raft_consensus.cc:2454] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.002605 12082 tablet_service.cc:1467] Processing CreateTablet for tablet fb074bc2833742dda1e8e68ca5f2c8e7 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240502 21:29:30.003144 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb074bc2833742dda1e8e68ca5f2c8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.003604 12295 tablet_bootstrap.cc:654] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.003170 12065 leader_election.cc:302] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:30.007067 12294 tablet_bootstrap.cc:654] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.007617 12159 tablet_service.cc:1467] Processing CreateTablet for tablet 04a7f75fa74945c09343353c01f4b31b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240502 21:29:30.008124 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a7f75fa74945c09343353c01f4b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.008893 12297 tablet_bootstrap.cc:492] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.009470 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 73d4e5a4db934406b9f995442ec6bcd1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240502 21:29:30.011281 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d4e5a4db934406b9f995442ec6bcd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.024753 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 060e5a3ff13f40629080ec7036eb39ef (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240502 21:29:30.027446 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e5a3ff13f40629080ec7036eb39ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.029850 12297 tablet_bootstrap.cc:654] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.038717 12090 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5655964c244669bd1b644c9399cdec (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240502 21:29:30.043006 12235 tablet_service.cc:1467] Processing CreateTablet for tablet d4fbdb33d94f4dd196663efa7f5b1cd7 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240502 21:29:30.043699 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fbdb33d94f4dd196663efa7f5b1cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.047597 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 528a557fdd90485d91077d2a61095878 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240502 21:29:30.051177 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528a557fdd90485d91077d2a61095878. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.061997 12309 raft_consensus.cc:2781] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.062134 12309 raft_consensus.cc:482] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.062214 12309 raft_consensus.cc:3037] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:30.062081 12065 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.185.67:34725, user_credentials={real_user=slave}} blocked reactor thread for 58901.1us
I20240502 21:29:30.063354 12297 tablet_bootstrap.cc:492] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.063517 12297 ts_tablet_manager.cc:1367] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.055s user 0.018s sys 0.011s
I20240502 21:29:30.064323 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5655964c244669bd1b644c9399cdec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.074319 12298 raft_consensus.cc:504] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.075184 12089 tablet_service.cc:1467] Processing CreateTablet for tablet 969594f6d4b248758a81cc2b1b5bccca (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240502 21:29:30.077857 12084 tablet_service.cc:1467] Processing CreateTablet for tablet 4a729e66312d4e7fb61c9f7ecfe8d271 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240502 21:29:30.078724 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafaa80ca77c464e948c967cc0a32055" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.079016 12176 raft_consensus.cc:3037] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.086864 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a729e66312d4e7fb61c9f7ecfe8d271. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.088266 12298 leader_election.cc:288] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.089040 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafaa80ca77c464e948c967cc0a32055" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.093283 12236 tablet_service.cc:1467] Processing CreateTablet for tablet af9adc5b5ce04374bf15c9088b4397d8 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240502 21:29:30.094374 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9adc5b5ce04374bf15c9088b4397d8. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 21:29:30.095257 12065 leader_election.cc:341] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.095985 12089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969594f6d4b248758a81cc2b1b5bccca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.102250 12241 tablet_service.cc:1467] Processing CreateTablet for tablet e8c09277218147c6a65b7d984a13c9b6 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240502 21:29:30.105222 12160 tablet_service.cc:1467] Processing CreateTablet for tablet 1b8307fd623a499fa05cd9eca238ddc2 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240502 21:29:30.106016 12160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8307fd623a499fa05cd9eca238ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.108168 12081 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbfbbbb346f413782d97580adfa82c1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240502 21:29:30.109999 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbfbbbb346f413782d97580adfa82c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.112565 12241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c09277218147c6a65b7d984a13c9b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.108209 12297 raft_consensus.cc:348] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.113582 12297 raft_consensus.cc:374] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.113688 12297 raft_consensus.cc:729] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.114084 12297 consensus_queue.cc:260] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.115486 12240 tablet_service.cc:1467] Processing CreateTablet for tablet 04a7f75fa74945c09343353c01f4b31b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240502 21:29:30.115931 12240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a7f75fa74945c09343353c01f4b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.116972 12295 tablet_bootstrap.cc:492] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.117120 12295 ts_tablet_manager.cc:1367] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.162s user 0.018s sys 0.034s
I20240502 21:29:30.118834 12085 tablet_service.cc:1467] Processing CreateTablet for tablet 40d1079e722d4ca0b4db86807f9d7a7b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240502 21:29:30.120198 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d1079e722d4ca0b4db86807f9d7a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.126387 12295 raft_consensus.cc:348] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.126626 12295 raft_consensus.cc:374] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.127138 12234 tablet_service.cc:1467] Processing CreateTablet for tablet 73d4e5a4db934406b9f995442ec6bcd1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240502 21:29:30.126700 12295 raft_consensus.cc:729] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.127795 12295 consensus_queue.cc:260] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.130936 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d4e5a4db934406b9f995442ec6bcd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.131484 12295 ts_tablet_manager.cc:1398] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20240502 21:29:30.131831 12295 tablet_bootstrap.cc:492] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.136670 12243 tablet_service.cc:1467] Processing CreateTablet for tablet 060e5a3ff13f40629080ec7036eb39ef (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240502 21:29:30.136718 12242 tablet_service.cc:1467] Processing CreateTablet for tablet 528a557fdd90485d91077d2a61095878 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240502 21:29:30.137199 12243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e5a3ff13f40629080ec7036eb39ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.137471 12242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528a557fdd90485d91077d2a61095878. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.138700 12309 raft_consensus.cc:504] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.139847 12297 ts_tablet_manager.cc:1398] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.076s user 0.002s sys 0.002s
I20240502 21:29:30.140208 12297 tablet_bootstrap.cc:492] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.140631 12238 tablet_service.cc:1467] Processing CreateTablet for tablet 1b8307fd623a499fa05cd9eca238ddc2 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240502 21:29:30.141011 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8307fd623a499fa05cd9eca238ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.141186 12164 tablet_service.cc:1467] Processing CreateTablet for tablet fb074bc2833742dda1e8e68ca5f2c8e7 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240502 21:29:30.141583 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb074bc2833742dda1e8e68ca5f2c8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.142706 12301 raft_consensus.cc:482] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.142855 12301 raft_consensus.cc:504] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.143792 12301 leader_election.cc:288] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.144982 12309 leader_election.cc:288] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.145386 12086 tablet_service.cc:1467] Processing CreateTablet for tablet b5cc5520fead4142819e3842fd573287 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240502 21:29:30.145908 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9758cbdd5b845d6883c6e34d0e3f6a6" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.146104 12252 raft_consensus.cc:3037] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.148423 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cc5520fead4142819e3842fd573287. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.153230 12176 raft_consensus.cc:2454] T dafaa80ca77c464e948c967cc0a32055 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.153481 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9758cbdd5b845d6883c6e34d0e3f6a6" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.155660 12309 raft_consensus.cc:482] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.155853 12309 raft_consensus.cc:504] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.157413 12309 leader_election.cc:288] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.157173 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:30.160115 12065 leader_election.cc:341] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.161448 12294 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.161620 12294 ts_tablet_manager.cc:1367] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.223s user 0.026s sys 0.031s
I20240502 21:29:30.168285 12239 tablet_service.cc:1467] Processing CreateTablet for tablet fb074bc2833742dda1e8e68ca5f2c8e7 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240502 21:29:30.168980 12239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb074bc2833742dda1e8e68ca5f2c8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.170957 12087 tablet_service.cc:1467] Processing CreateTablet for tablet b211abd4317f4be485f9db73fd55629e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240502 21:29:30.171603 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b211abd4317f4be485f9db73fd55629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.182451 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5655964c244669bd1b644c9399cdec (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240502 21:29:30.189004 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5655964c244669bd1b644c9399cdec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.196357 12066 leader_election.cc:302] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.197782 12309 raft_consensus.cc:2781] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.198292 12309 raft_consensus.cc:686] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
W20240502 21:29:30.198973 12066 leader_election.cc:341] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.198966 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.199187 12177 raft_consensus.cc:2454] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.198041 12294 raft_consensus.cc:348] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.199630 12066 leader_election.cc:302] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.199683 12294 raft_consensus.cc:374] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.199779 12294 raft_consensus.cc:729] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.199860 12298 raft_consensus.cc:2781] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.199931 12298 raft_consensus.cc:482] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.200073 12298 raft_consensus.cc:3037] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.200073 12294 consensus_queue.cc:260] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.207389 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 969594f6d4b248758a81cc2b1b5bccca (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240502 21:29:30.207882 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969594f6d4b248758a81cc2b1b5bccca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.209456 12295 tablet_bootstrap.cc:654] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.198204 12083 tablet_service.cc:1467] Processing CreateTablet for tablet 1efbb27ede324d8d9005518cf8a66360 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20240502 21:29:30.212137 12046 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240502 21:29:30.198602 12309 consensus_queue.cc:237] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.213706 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1efbb27ede324d8d9005518cf8a66360. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.214905 12297 tablet_bootstrap.cc:654] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.220762 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 4a729e66312d4e7fb61c9f7ecfe8d271 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240502 21:29:30.221247 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a729e66312d4e7fb61c9f7ecfe8d271. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.221938 12166 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbfbbbb346f413782d97580adfa82c1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240502 21:29:30.222450 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbfbbbb346f413782d97580adfa82c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.223922 12082 tablet_service.cc:1467] Processing CreateTablet for tablet dd6c003ea2884ca9a1827814a4930742 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240502 21:29:30.227504 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6c003ea2884ca9a1827814a4930742. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.230134 12158 tablet_service.cc:1467] Processing CreateTablet for tablet 40d1079e722d4ca0b4db86807f9d7a7b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240502 21:29:30.231046 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d1079e722d4ca0b4db86807f9d7a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.234097 12236 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5655964c244669bd1b644c9399cdec (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240502 21:29:30.245227 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5655964c244669bd1b644c9399cdec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.246629 12294 ts_tablet_manager.cc:1398] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.083s user 0.018s sys 0.017s
I20240502 21:29:30.246953 12294 tablet_bootstrap.cc:492] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.247411 11994 catalog_manager.cc:5484] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.248613 12252 raft_consensus.cc:2454] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.249106 12065 leader_election.cc:302] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:30.250173 12309 raft_consensus.cc:2781] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.250353 12309 raft_consensus.cc:686] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:30.260592 12309 consensus_queue.cc:237] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.270365 12235 tablet_service.cc:1467] Processing CreateTablet for tablet 969594f6d4b248758a81cc2b1b5bccca (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240502 21:29:30.271122 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 969594f6d4b248758a81cc2b1b5bccca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.276890 12161 tablet_service.cc:1467] Processing CreateTablet for tablet b5cc5520fead4142819e3842fd573287 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240502 21:29:30.278230 12167 tablet_service.cc:1467] Processing CreateTablet for tablet b211abd4317f4be485f9db73fd55629e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240502 21:29:30.278806 12167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b211abd4317f4be485f9db73fd55629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.282032 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cc5520fead4142819e3842fd573287. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.300366 12298 raft_consensus.cc:504] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.301473 12298 leader_election.cc:288] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.301951 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
W20240502 21:29:30.302512 12065 leader_election.cc:341] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.308817 12236 tablet_service.cc:1467] Processing CreateTablet for tablet 4a729e66312d4e7fb61c9f7ecfe8d271 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240502 21:29:30.309475 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a729e66312d4e7fb61c9f7ecfe8d271. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.311342 12308 raft_consensus.cc:482] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.311571 12308 raft_consensus.cc:504] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.312606 12308 leader_election.cc:288] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.315637 12242 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbfbbbb346f413782d97580adfa82c1 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240502 21:29:30.331712 12242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbfbbbb346f413782d97580adfa82c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.320359 12082 tablet_service.cc:1467] Processing CreateTablet for tablet a160b9e9063a4b95938cf7a5c944d77a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240502 21:29:30.332849 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a160b9e9063a4b95938cf7a5c944d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.327021 12087 tablet_service.cc:1467] Processing CreateTablet for tablet 19293e7da2774ce4aa900e851931b6ae (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240502 21:29:30.328548 12088 tablet_service.cc:1467] Processing CreateTablet for tablet df964090d307434981236aa967944cee (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240502 21:29:30.334738 12239 tablet_service.cc:1467] Processing CreateTablet for tablet b211abd4317f4be485f9db73fd55629e (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240502 21:29:30.336910 12239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b211abd4317f4be485f9db73fd55629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.337917 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df964090d307434981236aa967944cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.338155 12234 tablet_service.cc:1467] Processing CreateTablet for tablet 40d1079e722d4ca0b4db86807f9d7a7b (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240502 21:29:30.338663 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d1079e722d4ca0b4db86807f9d7a7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.338860 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19293e7da2774ce4aa900e851931b6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.335705 12237 tablet_service.cc:1467] Processing CreateTablet for tablet 1efbb27ede324d8d9005518cf8a66360 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240502 21:29:30.339375 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1efbb27ede324d8d9005518cf8a66360. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.330399 12159 tablet_service.cc:1467] Processing CreateTablet for tablet 1efbb27ede324d8d9005518cf8a66360 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240502 21:29:30.341133 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1efbb27ede324d8d9005518cf8a66360. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.326892 12241 tablet_service.cc:1467] Processing CreateTablet for tablet b5cc5520fead4142819e3842fd573287 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240502 21:29:30.341787 12241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cc5520fead4142819e3842fd573287. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.349282 12236 tablet_service.cc:1467] Processing CreateTablet for tablet dd6c003ea2884ca9a1827814a4930742 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240502 21:29:30.349897 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6c003ea2884ca9a1827814a4930742. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.351028 12298 raft_consensus.cc:482] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.351163 12298 raft_consensus.cc:504] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.351812 12298 leader_election.cc:288] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.366411 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.366717 12177 raft_consensus.cc:3037] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.368808 12235 tablet_service.cc:1467] Processing CreateTablet for tablet a160b9e9063a4b95938cf7a5c944d77a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240502 21:29:30.369530 12090 tablet_service.cc:1467] Processing CreateTablet for tablet f5ef70a542484291bfb51c87b4d2f838 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240502 21:29:30.369531 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a160b9e9063a4b95938cf7a5c944d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.379066 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ef70a542484291bfb51c87b4d2f838. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.380438 12301 raft_consensus.cc:482] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.380620 12301 raft_consensus.cc:504] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.381819 12081 tablet_service.cc:1467] Processing CreateTablet for tablet b158108cfdab4223b8f9d0226578de60 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240502 21:29:30.382401 12081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158108cfdab4223b8f9d0226578de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.383411 12294 tablet_bootstrap.cc:654] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.376557 12163 tablet_service.cc:1467] Processing CreateTablet for tablet dd6c003ea2884ca9a1827814a4930742 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240502 21:29:30.384788 12301 leader_election.cc:288] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.385228 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6c003ea2884ca9a1827814a4930742. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.386225 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77279927ddf545e9a3216606b9bab54d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:30.387912 12066 leader_election.cc:341] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.392975 12240 tablet_service.cc:1467] Processing CreateTablet for tablet 19293e7da2774ce4aa900e851931b6ae (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240502 21:29:30.393608 12240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19293e7da2774ce4aa900e851931b6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.395032 12236 tablet_service.cc:1467] Processing CreateTablet for tablet df964090d307434981236aa967944cee (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240502 21:29:30.395581 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df964090d307434981236aa967944cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.406936 12243 tablet_service.cc:1467] Processing CreateTablet for tablet f5ef70a542484291bfb51c87b4d2f838 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240502 21:29:30.407452 12243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ef70a542484291bfb51c87b4d2f838. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.410894 12160 tablet_service.cc:1467] Processing CreateTablet for tablet a160b9e9063a4b95938cf7a5c944d77a (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240502 21:29:30.411367 12160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a160b9e9063a4b95938cf7a5c944d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.412765 12295 tablet_bootstrap.cc:492] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.412904 12295 ts_tablet_manager.cc:1367] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.281s user 0.014s sys 0.012s
I20240502 21:29:30.413672 12295 raft_consensus.cc:348] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.413861 12295 raft_consensus.cc:374] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.413908 12295 raft_consensus.cc:729] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.414120 12295 consensus_queue.cc:260] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.414712 12295 ts_tablet_manager.cc:1398] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:30.415014 12295 tablet_bootstrap.cc:492] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.417485 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 19293e7da2774ce4aa900e851931b6ae (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240502 21:29:30.417917 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19293e7da2774ce4aa900e851931b6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.420609 12309 raft_consensus.cc:482] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.420812 12309 raft_consensus.cc:504] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.425657 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c069ca9b95cc4e8ea75c71ea6cefdcc3" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.426327 12309 leader_election.cc:288] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:30.426501 12066 leader_election.cc:341] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.430680 12297 tablet_bootstrap.cc:492] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.430856 12297 ts_tablet_manager.cc:1367] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.291s user 0.050s sys 0.068s
I20240502 21:29:30.431631 12161 tablet_service.cc:1467] Processing CreateTablet for tablet df964090d307434981236aa967944cee (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240502 21:29:30.432044 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df964090d307434981236aa967944cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.430925 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c069ca9b95cc4e8ea75c71ea6cefdcc3" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:30.433485 12065 leader_election.cc:341] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.433621 12065 leader_election.cc:302] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.433862 12309 raft_consensus.cc:2726] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:30.435657 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.429916 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77279927ddf545e9a3216606b9bab54d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.442016 12162 tablet_service.cc:1467] Processing CreateTablet for tablet f5ef70a542484291bfb51c87b4d2f838 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240502 21:29:30.444485 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ef70a542484291bfb51c87b4d2f838. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.437249 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56cd8f7bdee468b846b4f51e86e1f27" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:30.452364 12065 leader_election.cc:341] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.454859 12065 leader_election.cc:302] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.455386 12309 raft_consensus.cc:2726] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 21:29:30.456045 12142 leader_election.cc:341] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.463733 12238 tablet_service.cc:1467] Processing CreateTablet for tablet b158108cfdab4223b8f9d0226578de60 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240502 21:29:30.464453 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158108cfdab4223b8f9d0226578de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.465909 12295 tablet_bootstrap.cc:654] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
W20240502 21:29:30.466509 12142 leader_election.cc:341] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.466697 12297 raft_consensus.cc:348] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.468997 12297 raft_consensus.cc:374] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.469063 12297 raft_consensus.cc:729] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.472712 12165 tablet_service.cc:1467] Processing CreateTablet for tablet b158108cfdab4223b8f9d0226578de60 (DEFAULT_TABLE table=test-workload [id=ac5c3611cbbc479f88474c64c2be8100]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240502 21:29:30.473157 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158108cfdab4223b8f9d0226578de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:30.479140 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56cd8f7bdee468b846b4f51e86e1f27" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.479099 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.487375 12100 raft_consensus.cc:2379] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:30.481941 12297 consensus_queue.cc:260] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.489434 12297 ts_tablet_manager.cc:1398] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.058s user 0.001s sys 0.001s
I20240502 21:29:30.489728 12297 tablet_bootstrap.cc:492] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.497645 12177 raft_consensus.cc:2454] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.500928 12066 leader_election.cc:302] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.501322 12309 raft_consensus.cc:2781] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.501478 12309 raft_consensus.cc:686] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:30.497296 12140 leader_election.cc:302] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.500093 12090 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39156 (request call id 57) took 1007 ms. Trace:
I20240502 21:29:30.501878 12301 raft_consensus.cc:2726] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 21:29:30.502071 12140 leader_election.cc:341] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.502166 12140 leader_election.cc:302] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.502349 12301 raft_consensus.cc:2726] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:30.503321 12297 tablet_bootstrap.cc:654] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.501775 12309 consensus_queue.cc:237] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.501937 12090 rpcz_store.cc:276] 0502 21:29:29.492257 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.340641 (+848384us) service_pool.cc:225] Handling call
0502 21:29:30.369860 (+ 29219us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.369899 (+ 39us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.419679 (+ 49780us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.499864 (+ 80185us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.500065 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.508486 12086 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39156 (request call id 50) took 1019 ms. Trace:
I20240502 21:29:30.508569 12086 rpcz_store.cc:276] 0502 21:29:29.488509 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.141269 (+652760us) service_pool.cc:225] Handling call
0502 21:29:30.145716 (+ 4447us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.145765 (+ 49us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.325091 (+179326us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.508339 (+183248us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.508456 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.513960 12294 tablet_bootstrap.cc:492] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.514927 12294 ts_tablet_manager.cc:1367] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.268s user 0.016s sys 0.041s
I20240502 21:29:30.529978 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bec182656a4200b4e9938e861ab21f" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.530360 12177 raft_consensus.cc:2454] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.533480 12219 leader_election.cc:302] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.533865 12308 raft_consensus.cc:2781] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.533982 12308 raft_consensus.cc:482] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.534068 12308 raft_consensus.cc:3037] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.533962 12294 raft_consensus.cc:348] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.534184 12294 raft_consensus.cc:374] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.534245 12294 raft_consensus.cc:729] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.539283 12294 consensus_queue.cc:260] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.551787 12294 ts_tablet_manager.cc:1398] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.036s user 0.001s sys 0.001s
I20240502 21:29:30.552181 12294 tablet_bootstrap.cc:492] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.562387 12082 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39156 (request call id 54) took 1072 ms. Trace:
I20240502 21:29:30.562556 12082 rpcz_store.cc:276] 0502 21:29:29.490376 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.319885 (+829509us) service_pool.cc:225] Handling call
0502 21:29:30.332630 (+ 12745us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.332664 (+ 34us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.481113 (+148449us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.562087 (+ 80974us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.562346 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.563338 12167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55152 (request call id 51) took 1017 ms. Trace:
I20240502 21:29:30.563437 12167 rpcz_store.cc:276] 0502 21:29:29.545707 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.277797 (+732090us) service_pool.cc:225] Handling call
0502 21:29:30.278545 (+ 748us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.278591 (+ 46us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.476966 (+198375us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.563196 (+ 86230us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.563320 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.567524 12295 tablet_bootstrap.cc:492] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.567715 12295 ts_tablet_manager.cc:1367] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.153s user 0.015s sys 0.019s
I20240502 21:29:30.572211 12088 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39156 (request call id 56) took 1080 ms. Trace:
I20240502 21:29:30.572332 12088 rpcz_store.cc:276] 0502 21:29:29.492137 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.328239 (+836102us) service_pool.cc:225] Handling call
0502 21:29:30.336443 (+ 8204us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.336472 (+ 29us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.464387 (+127915us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.571985 (+107598us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.572183 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.574446 12081 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39156 (request call id 58) took 1079 ms. Trace:
I20240502 21:29:30.574553 12081 rpcz_store.cc:276] 0502 21:29:29.495086 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.378343 (+883257us) service_pool.cc:225] Handling call
0502 21:29:30.382148 (+ 3805us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.382198 (+ 50us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.515498 (+133300us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.574284 (+ 58786us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.574422 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.575011 12332 raft_consensus.cc:482] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.576306 12159 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55152 (request call id 52) took 1030 ms. Trace:
I20240502 21:29:30.576690 12087 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39156 (request call id 55) took 1086 ms. Trace:
I20240502 21:29:30.576761 12087 rpcz_store.cc:276] 0502 21:29:29.490478 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.326652 (+836174us) service_pool.cc:225] Handling call
0502 21:29:30.336201 (+ 9549us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.336247 (+ 46us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.484449 (+148202us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.576531 (+ 92082us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.576668 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.576723 12159 rpcz_store.cc:276] 0502 21:29:29.545747 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.330129 (+784382us) service_pool.cc:225] Handling call
0502 21:29:30.340939 (+ 10810us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.340969 (+ 30us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.476679 (+135710us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.576179 (+ 99500us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.576288 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.578802 12234 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 49) took 1016 ms. Trace:
I20240502 21:29:30.578886 12234 rpcz_store.cc:276] 0502 21:29:29.562023 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.322970 (+760947us) service_pool.cc:225] Handling call
0502 21:29:30.338452 (+ 15482us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.338490 (+ 38us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.475924 (+137434us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.578683 (+102759us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.578784 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.580789 12240 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 55) took 1018 ms. Trace:
I20240502 21:29:30.580904 12240 rpcz_store.cc:276] 0502 21:29:29.562513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.388794 (+826281us) service_pool.cc:225] Handling call
0502 21:29:30.393363 (+ 4569us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.393414 (+ 51us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.494214 (+100800us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.580671 (+ 86457us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.580772 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.580785 11994 catalog_manager.cc:5484] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.581533 11994 catalog_manager.cc:5484] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.581547 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bec182656a4200b4e9938e861ab21f" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
W20240502 21:29:30.582952 12216 leader_election.cc:341] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.577900 12332 raft_consensus.cc:504] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.587538 12297 tablet_bootstrap.cc:492] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.587675 12160 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55152 (request call id 54) took 1040 ms. Trace:
I20240502 21:29:30.587761 12160 rpcz_store.cc:276] 0502 21:29:29.546909 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.410591 (+863682us) service_pool.cc:225] Handling call
0502 21:29:30.411165 (+ 574us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.411206 (+ 41us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.530664 (+119458us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.587486 (+ 56822us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.587655 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.588446 12297 ts_tablet_manager.cc:1367] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.099s user 0.018s sys 0.035s
I20240502 21:29:30.588553 12161 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55152 (request call id 56) took 1033 ms. Trace:
I20240502 21:29:30.588654 12161 rpcz_store.cc:276] 0502 21:29:29.555295 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.431404 (+876109us) service_pool.cc:225] Handling call
0502 21:29:30.431832 (+ 428us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.431865 (+ 33us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.569106 (+137241us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.588368 (+ 19262us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.588527 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.589994 12242 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 48) took 1033 ms. Trace:
I20240502 21:29:30.590080 12242 rpcz_store.cc:276] 0502 21:29:29.561865 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.315161 (+753296us) service_pool.cc:225] Handling call
0502 21:29:30.331429 (+ 16268us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.331484 (+ 55us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.507461 (+175977us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.589234 (+ 81773us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.589970 (+ 736us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.591387 12241 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 50) took 1029 ms. Trace:
I20240502 21:29:30.591467 12241 rpcz_store.cc:276] 0502 21:29:29.562131 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.326580 (+764449us) service_pool.cc:225] Handling call
0502 21:29:30.341609 (+ 15029us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.341644 (+ 35us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.489147 (+147503us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.591222 (+102075us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.591363 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.592276 12295 raft_consensus.cc:348] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.592511 12295 raft_consensus.cc:374] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.592588 12295 raft_consensus.cc:729] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.592826 12295 consensus_queue.cc:260] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.594713 12295 ts_tablet_manager.cc:1398] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.027s user 0.002s sys 0.000s
I20240502 21:29:30.595075 12295 tablet_bootstrap.cc:492] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.595327 12236 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 56) took 1030 ms. Trace:
I20240502 21:29:30.595410 12236 rpcz_store.cc:276] 0502 21:29:29.564894 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.394614 (+829720us) service_pool.cc:225] Handling call
0502 21:29:30.395358 (+ 744us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.395401 (+ 43us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.510762 (+115361us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.595133 (+ 84371us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.595303 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.596228 12332 leader_election.cc:288] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.596794 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.597584 12243 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 57) took 1032 ms. Trace:
I20240502 21:29:30.597658 12243 rpcz_store.cc:276] 0502 21:29:29.564997 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.406605 (+841608us) service_pool.cc:225] Handling call
0502 21:29:30.407245 (+ 640us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.407282 (+ 37us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.509100 (+101818us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.597464 (+ 88364us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.597566 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.596114 12308 raft_consensus.cc:504] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.598829 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d05591283c24d0d9e6de5ad16ab504d" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.598974 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bec182656a4200b4e9938e861ab21f" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.599094 12177 raft_consensus.cc:2379] T 6d05591283c24d0d9e6de5ad16ab504d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:30.599013 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bec182656a4200b4e9938e861ab21f" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.599211 12176 raft_consensus.cc:3037] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:30.599440 12216 leader_election.cc:341] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.599809 12308 leader_election.cc:288] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.599776 12219 leader_election.cc:302] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:30.600916 12308 raft_consensus.cc:3037] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.601951 12294 tablet_bootstrap.cc:654] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.603066 12238 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52824 (request call id 58) took 1038 ms. Trace:
I20240502 21:29:30.603147 12238 rpcz_store.cc:276] 0502 21:29:29.565073 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.462741 (+897668us) service_pool.cc:225] Handling call
0502 21:29:30.464236 (+ 1495us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.464278 (+ 42us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.582539 (+118261us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.602917 (+ 20378us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.603047 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.604383 12162 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55152 (request call id 57) took 1049 ms. Trace:
I20240502 21:29:30.604480 12162 rpcz_store.cc:276] 0502 21:29:29.555383 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.440644 (+885261us) service_pool.cc:225] Handling call
0502 21:29:30.444284 (+ 3640us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.444325 (+ 41us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.559362 (+115037us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.604213 (+ 44851us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.604351 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.605619 12297 raft_consensus.cc:348] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.605813 12176 raft_consensus.cc:2454] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.606384 12297 raft_consensus.cc:374] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.606552 12297 raft_consensus.cc:729] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.606573 12219 leader_election.cc:302] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0
I20240502 21:29:30.607082 12297 consensus_queue.cc:260] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.607519 12332 raft_consensus.cc:2781] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.607991 12165 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55152 (request call id 58) took 1052 ms. Trace:
I20240502 21:29:30.608089 12165 rpcz_store.cc:276] 0502 21:29:29.555443 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 21:29:30.470212 (+914769us) service_pool.cc:225] Handling call
0502 21:29:30.472973 (+ 2761us) ts_tablet_manager.cc:642] Acquired tablet manager lock
0502 21:29:30.473013 (+ 40us) ts_tablet_manager.cc:655] Creating new metadata...
0502 21:29:30.590927 (+117914us) tablet_metadata.cc:641] Metadata flushed
0502 21:29:30.607834 (+ 16907us) tablet_replica.cc:205] Creating consensus instance
0502 21:29:30.607941 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20240502 21:29:30.611254 12308 raft_consensus.cc:2726] T 6d05591283c24d0d9e6de5ad16ab504d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:30.611379 12297 ts_tablet_manager.cc:1398] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.023s user 0.001s sys 0.002s
I20240502 21:29:30.611734 12297 tablet_bootstrap.cc:492] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.612707 12295 tablet_bootstrap.cc:654] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.612756 12294 tablet_bootstrap.cc:492] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.612901 12294 ts_tablet_manager.cc:1367] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.061s user 0.005s sys 0.002s
I20240502 21:29:30.613638 12297 tablet_bootstrap.cc:654] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.613715 12294 raft_consensus.cc:348] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.613907 12332 raft_consensus.cc:686] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:30.613934 12294 raft_consensus.cc:374] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.614149 12294 raft_consensus.cc:729] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.614454 12294 consensus_queue.cc:260] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.614358 12332 consensus_queue.cc:237] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.615190 12295 tablet_bootstrap.cc:492] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.615682 12295 ts_tablet_manager.cc:1367] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20240502 21:29:30.616458 12295 raft_consensus.cc:348] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.616736 12295 raft_consensus.cc:374] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.616845 12295 raft_consensus.cc:729] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.615283 12294 ts_tablet_manager.cc:1398] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20240502 21:29:30.617388 12294 tablet_bootstrap.cc:492] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.619354 12295 consensus_queue.cc:260] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.621387 12295 ts_tablet_manager.cc:1398] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 21:29:30.621698 12295 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.623562 12295 tablet_bootstrap.cc:654] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.623200 11995 catalog_manager.cc:5484] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:30.623847 12297 tablet_bootstrap.cc:492] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.624039 12297 ts_tablet_manager.cc:1367] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.007s
I20240502 21:29:30.625041 12294 tablet_bootstrap.cc:654] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.624909 12297 raft_consensus.cc:348] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.625144 12297 raft_consensus.cc:374] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.625227 12297 raft_consensus.cc:729] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.625484 12297 consensus_queue.cc:260] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.626063 12297 ts_tablet_manager.cc:1398] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:30.626389 12297 tablet_bootstrap.cc:492] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.626987 12295 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.627130 12295 ts_tablet_manager.cc:1367] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 21:29:30.627790 12295 raft_consensus.cc:348] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.627974 12295 raft_consensus.cc:374] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.628041 12295 raft_consensus.cc:729] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.628345 12297 tablet_bootstrap.cc:654] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.628276 12295 consensus_queue.cc:260] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.629009 12295 ts_tablet_manager.cc:1398] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:30.629738 12294 tablet_bootstrap.cc:492] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.629920 12294 ts_tablet_manager.cc:1367] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.001s
I20240502 21:29:30.630774 12294 raft_consensus.cc:348] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.630971 12294 raft_consensus.cc:374] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.631034 12294 raft_consensus.cc:729] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.631321 12294 consensus_queue.cc:260] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.632524 12297 tablet_bootstrap.cc:492] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.632678 12297 ts_tablet_manager.cc:1367] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 21:29:30.632810 12295 tablet_bootstrap.cc:492] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.633461 12297 raft_consensus.cc:348] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.633679 12297 raft_consensus.cc:374] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.633760 12297 raft_consensus.cc:729] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.634081 12297 consensus_queue.cc:260] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.634760 12295 tablet_bootstrap.cc:654] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.634883 12297 ts_tablet_manager.cc:1398] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:30.635207 12297 tablet_bootstrap.cc:492] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.632675 12294 ts_tablet_manager.cc:1398] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:30.636947 12294 tablet_bootstrap.cc:492] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.637110 12297 tablet_bootstrap.cc:654] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.638738 12294 tablet_bootstrap.cc:654] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.639439 12297 tablet_bootstrap.cc:492] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.639596 12297 ts_tablet_manager.cc:1367] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:30.640568 12297 raft_consensus.cc:348] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.640771 12297 raft_consensus.cc:374] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.640794 12294 tablet_bootstrap.cc:492] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.640836 12297 raft_consensus.cc:729] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.640944 12294 ts_tablet_manager.cc:1367] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:30.641080 12297 consensus_queue.cc:260] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.641278 12295 tablet_bootstrap.cc:492] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.641402 12295 ts_tablet_manager.cc:1367] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20240502 21:29:30.641749 12297 ts_tablet_manager.cc:1398] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:30.641649 12294 raft_consensus.cc:348] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.641847 12294 raft_consensus.cc:374] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.641912 12294 raft_consensus.cc:729] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.642036 12297 tablet_bootstrap.cc:492] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.642115 12294 consensus_queue.cc:260] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.642648 12294 ts_tablet_manager.cc:1398] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:30.643086 12294 tablet_bootstrap.cc:492] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.643854 12297 tablet_bootstrap.cc:654] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.644948 12294 tablet_bootstrap.cc:654] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.642305 12295 raft_consensus.cc:348] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.648528 12295 raft_consensus.cc:374] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.648633 12295 raft_consensus.cc:729] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.648980 12295 consensus_queue.cc:260] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.649222 12294 tablet_bootstrap.cc:492] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.649360 12294 ts_tablet_manager.cc:1367] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20240502 21:29:30.649861 12295 ts_tablet_manager.cc:1398] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20240502 21:29:30.650218 12295 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.650159 12294 raft_consensus.cc:348] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.650424 12294 raft_consensus.cc:374] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.650507 12294 raft_consensus.cc:729] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.650768 12294 consensus_queue.cc:260] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.651234 12294 ts_tablet_manager.cc:1398] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:30.651480 12294 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.653275 12294 tablet_bootstrap.cc:654] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.655572 12294 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.655720 12294 ts_tablet_manager.cc:1367] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:30.656455 12294 raft_consensus.cc:348] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.656620 12294 raft_consensus.cc:374] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.656683 12294 raft_consensus.cc:729] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.656908 12294 consensus_queue.cc:260] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.657410 12294 ts_tablet_manager.cc:1398] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 21:29:30.657666 12294 tablet_bootstrap.cc:492] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.658998 12294 tablet_bootstrap.cc:654] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.660658 12295 tablet_bootstrap.cc:654] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.661679 12294 tablet_bootstrap.cc:492] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.661861 12294 ts_tablet_manager.cc:1367] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240502 21:29:30.662495 12294 raft_consensus.cc:348] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.662729 12294 raft_consensus.cc:374] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.662833 12294 raft_consensus.cc:729] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.663105 12294 consensus_queue.cc:260] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.663614 12294 ts_tablet_manager.cc:1398] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:30.663919 12294 tablet_bootstrap.cc:492] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.665314 12294 tablet_bootstrap.cc:654] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.667304 12294 tablet_bootstrap.cc:492] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.667479 12294 ts_tablet_manager.cc:1367] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:30.668277 12294 raft_consensus.cc:348] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.668496 12294 raft_consensus.cc:374] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.668614 12294 raft_consensus.cc:729] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.668861 12294 consensus_queue.cc:260] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.669416 12294 ts_tablet_manager.cc:1398] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:30.669716 12294 tablet_bootstrap.cc:492] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.671093 12294 tablet_bootstrap.cc:654] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.673976 12294 tablet_bootstrap.cc:492] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.674189 12294 ts_tablet_manager.cc:1367] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:30.674212 12297 tablet_bootstrap.cc:492] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.674499 12297 ts_tablet_manager.cc:1367] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.032s user 0.021s sys 0.011s
I20240502 21:29:30.674647 12309 raft_consensus.cc:482] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.674818 12309 raft_consensus.cc:504] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.675179 12294 raft_consensus.cc:348] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.675410 12294 raft_consensus.cc:374] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.675360 12297 raft_consensus.cc:348] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.675575 12294 raft_consensus.cc:729] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.675640 12297 raft_consensus.cc:374] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.675640 12295 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.675760 12297 raft_consensus.cc:729] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.676010 12295 ts_tablet_manager.cc:1367] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.002s
I20240502 21:29:30.675866 12294 consensus_queue.cc:260] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.676081 12297 consensus_queue.cc:260] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.676530 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe982f82c003411e9f9dda42378399f8" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.676651 12294 ts_tablet_manager.cc:1398] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:30.676748 12297 ts_tablet_manager.cc:1398] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:30.677042 12294 tablet_bootstrap.cc:492] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.677075 12297 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.677009 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe982f82c003411e9f9dda42378399f8" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.677279 12295 raft_consensus.cc:348] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.677541 12295 raft_consensus.cc:374] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.677700 12295 raft_consensus.cc:729] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.678030 12295 consensus_queue.cc:260] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
W20240502 21:29:30.678602 12065 leader_election.cc:341] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.679068 12294 tablet_bootstrap.cc:654] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.679073 12297 tablet_bootstrap.cc:654] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
W20240502 21:29:30.679181 12066 leader_election.cc:341] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.679302 12066 leader_election.cc:302] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.679587 12298 raft_consensus.cc:2726] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:30.679764 12309 leader_election.cc:288] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.683302 12295 ts_tablet_manager.cc:1398] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20240502 21:29:30.685382 12295 tablet_bootstrap.cc:492] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.688105 12295 tablet_bootstrap.cc:654] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.688414 12297 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.688616 12297 ts_tablet_manager.cc:1367] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20240502 21:29:30.688699 12294 tablet_bootstrap.cc:492] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.688974 12309 raft_consensus.cc:482] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.689203 12309 raft_consensus.cc:504] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.689610 12297 raft_consensus.cc:348] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.689806 12297 raft_consensus.cc:374] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.689937 12297 raft_consensus.cc:729] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.690116 12309 leader_election.cc:288] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.688974 12294 ts_tablet_manager.cc:1367] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20240502 21:29:30.690208 12297 consensus_queue.cc:260] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.690834 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56cd8f7bdee468b846b4f51e86e1f27" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.691037 12297 ts_tablet_manager.cc:1398] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:30.691193 12253 raft_consensus.cc:2454] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.691748 12297 tablet_bootstrap.cc:492] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.691676 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56cd8f7bdee468b846b4f51e86e1f27" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.691936 12176 raft_consensus.cc:2454] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.692035 12295 tablet_bootstrap.cc:492] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.691874 12294 raft_consensus.cc:348] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.692157 12294 raft_consensus.cc:374] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.692248 12295 ts_tablet_manager.cc:1367] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20240502 21:29:30.692281 12294 raft_consensus.cc:729] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.692517 12294 consensus_queue.cc:260] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.693148 12295 raft_consensus.cc:348] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.693452 12295 raft_consensus.cc:374] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.693542 12295 raft_consensus.cc:729] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.693902 12065 leader_election.cc:302] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.693814 12295 consensus_queue.cc:260] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.694067 12294 ts_tablet_manager.cc:1398] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:30.694275 12297 tablet_bootstrap.cc:654] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.694408 12294 tablet_bootstrap.cc:492] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.696803 12294 tablet_bootstrap.cc:654] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.696902 12295 ts_tablet_manager.cc:1398] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:29:30.697271 12295 tablet_bootstrap.cc:492] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.697520 12332 raft_consensus.cc:482] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.697681 12332 raft_consensus.cc:504] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.698460 12332 leader_election.cc:288] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.699126 12295 tablet_bootstrap.cc:654] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.699168 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77279927ddf545e9a3216606b9bab54d" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.699466 12176 raft_consensus.cc:2454] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.699436 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77279927ddf545e9a3216606b9bab54d" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.699674 12099 raft_consensus.cc:2454] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.700053 12219 leader_election.cc:302] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.700592 12332 raft_consensus.cc:2781] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.700757 12332 raft_consensus.cc:482] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.700887 12332 raft_consensus.cc:3037] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.705231 12332 raft_consensus.cc:504] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.706169 12309 raft_consensus.cc:2781] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.706333 12309 raft_consensus.cc:482] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.706483 12309 raft_consensus.cc:3037] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.706698 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77279927ddf545e9a3216606b9bab54d" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.706992 12099 raft_consensus.cc:3037] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.710044 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77279927ddf545e9a3216606b9bab54d" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.710381 12176 raft_consensus.cc:3037] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.710685 12309 raft_consensus.cc:504] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.710947 12099 raft_consensus.cc:2454] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.711479 12216 leader_election.cc:302] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.711834 12335 raft_consensus.cc:2781] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.711997 12332 leader_election.cc:288] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.712225 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56cd8f7bdee468b846b4f51e86e1f27" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.712055 12335 raft_consensus.cc:686] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:30.713076 12309 leader_election.cc:288] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.713359 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56cd8f7bdee468b846b4f51e86e1f27" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.713503 12177 raft_consensus.cc:3037] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.713395 12335 consensus_queue.cc:237] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.715353 12253 raft_consensus.cc:3037] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.715776 12295 tablet_bootstrap.cc:492] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.716015 12295 ts_tablet_manager.cc:1367] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.009s
I20240502 21:29:30.716964 12295 raft_consensus.cc:348] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.717193 12295 raft_consensus.cc:374] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.717293 12295 raft_consensus.cc:729] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.717571 12295 consensus_queue.cc:260] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.718076 11994 catalog_manager.cc:5484] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:30.718839 12176 raft_consensus.cc:2454] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.720667 12295 ts_tablet_manager.cc:1398] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 21:29:30.721050 12295 tablet_bootstrap.cc:492] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.722296 12177 raft_consensus.cc:2454] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.722815 12301 raft_consensus.cc:482] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.723058 12066 leader_election.cc:302] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.722983 12301 raft_consensus.cc:504] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.723342 12309 raft_consensus.cc:2781] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.723503 12309 raft_consensus.cc:686] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:30.723763 12301 leader_election.cc:288] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.723822 12309 consensus_queue.cc:237] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.724267 12295 tablet_bootstrap.cc:654] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.724193 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9758cbdd5b845d6883c6e34d0e3f6a6" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.726183 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9758cbdd5b845d6883c6e34d0e3f6a6" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.726579 12251 raft_consensus.cc:2379] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:30.729441 12142 leader_election.cc:302] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.730201 12301 raft_consensus.cc:3037] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.736017 12253 raft_consensus.cc:2454] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.739101 12301 raft_consensus.cc:2726] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:30.741988 12301 raft_consensus.cc:482] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.743603 12298 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:30.743737 12301 raft_consensus.cc:504] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.746037 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0b9621b33c4afcb3842c10ac4c7148" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.746348 12099 raft_consensus.cc:2454] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:30.746927 12140 leader_election.cc:302] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.748086 12298 raft_consensus.cc:482] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.748337 12298 raft_consensus.cc:504] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.749207 12298 leader_election.cc:288] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.749972 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d88ae6514545bca16f20beb52b8e7b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.750288 12176 raft_consensus.cc:2454] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.750950 12066 leader_election.cc:302] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.751435 12298 raft_consensus.cc:2781] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.751559 12298 raft_consensus.cc:482] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.751701 12298 raft_consensus.cc:3037] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.759364 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d88ae6514545bca16f20beb52b8e7b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:30.760393 12065 leader_election.cc:341] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.764621 12301 leader_election.cc:288] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.765622 12301 raft_consensus.cc:2781] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.765851 12301 raft_consensus.cc:482] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.766775 12301 raft_consensus.cc:3037] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.766194 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0b9621b33c4afcb3842c10ac4c7148" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.772297 12253 raft_consensus.cc:2454] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:30.755792 12298 raft_consensus.cc:504] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.774472 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d88ae6514545bca16f20beb52b8e7b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.774729 12176 raft_consensus.cc:3037] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.778754 12176 raft_consensus.cc:2454] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.779723 12066 leader_election.cc:302] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.780093 12316 raft_consensus.cc:2781] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.782418 12298 leader_election.cc:288] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.782881 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d88ae6514545bca16f20beb52b8e7b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
W20240502 21:29:30.783715 12065 leader_election.cc:341] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.785578 12298 raft_consensus.cc:482] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.785946 12298 raft_consensus.cc:504] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.787842 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434ffa8e08cf45f6905eb6aaaab820ca" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:30.788571 12066 leader_election.cc:341] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.791739 12298 leader_election.cc:288] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.792210 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434ffa8e08cf45f6905eb6aaaab820ca" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:30.792887 12065 leader_election.cc:341] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:30.793083 12065 leader_election.cc:302] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:30.795311 12298 raft_consensus.cc:2726] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:30.796887 12294 tablet_bootstrap.cc:492] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.797052 12294 ts_tablet_manager.cc:1367] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.103s user 0.025s sys 0.045s
I20240502 21:29:30.797732 12297 tablet_bootstrap.cc:492] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.797868 12297 ts_tablet_manager.cc:1367] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.107s user 0.029s sys 0.049s
I20240502 21:29:30.797952 12294 raft_consensus.cc:348] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.798167 12294 raft_consensus.cc:374] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.798225 12294 raft_consensus.cc:729] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.798411 12294 consensus_queue.cc:260] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.798595 12297 raft_consensus.cc:348] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.798805 12297 raft_consensus.cc:374] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.798889 12297 raft_consensus.cc:729] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.799165 12297 consensus_queue.cc:260] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.803512 12297 ts_tablet_manager.cc:1398] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20240502 21:29:30.807644 12297 tablet_bootstrap.cc:492] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.810562 12295 tablet_bootstrap.cc:492] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.810755 12295 ts_tablet_manager.cc:1367] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.090s user 0.015s sys 0.015s
I20240502 21:29:30.814353 12298 consensus_queue.cc:1035] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:30.814894 12297 tablet_bootstrap.cc:654] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.814178 12366 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240502 21:29:30.816038 12316 raft_consensus.cc:686] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:30.816390 12316 consensus_queue.cc:237] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.817149 11994 catalog_manager.cc:5484] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.811846 12295 raft_consensus.cc:348] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.818678 12295 raft_consensus.cc:374] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.818756 12295 raft_consensus.cc:729] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.818950 12295 consensus_queue.cc:260] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.819510 12295 ts_tablet_manager.cc:1398] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20240502 21:29:30.821177 12251 raft_consensus.cc:3037] T dafaa80ca77c464e948c967cc0a32055 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.823736 12297 tablet_bootstrap.cc:492] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.823945 12297 ts_tablet_manager.cc:1367] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20240502 21:29:30.825033 12297 raft_consensus.cc:348] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.825314 12297 raft_consensus.cc:374] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.825381 12297 raft_consensus.cc:729] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.825572 12295 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.827194 12295 tablet_bootstrap.cc:654] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.825697 12297 consensus_queue.cc:260] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.828725 12297 ts_tablet_manager.cc:1398] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20240502 21:29:30.829128 12297 tablet_bootstrap.cc:492] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
W20240502 21:29:30.830300 12140 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.185.65:44241, user_credentials={real_user=slave}} blocked reactor thread for 83401.4us
I20240502 21:29:30.830916 12297 tablet_bootstrap.cc:654] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.831561 12367 consensus_queue.cc:1035] T dafaa80ca77c464e948c967cc0a32055 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:30.831776 12294 ts_tablet_manager.cc:1398] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.035s user 0.010s sys 0.023s
I20240502 21:29:30.832115 12294 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.833715 12294 tablet_bootstrap.cc:654] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.836565 12295 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.836762 12295 ts_tablet_manager.cc:1367] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.003s
I20240502 21:29:30.837716 12295 raft_consensus.cc:348] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.837992 12295 raft_consensus.cc:374] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.838059 12295 raft_consensus.cc:729] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.841671 12294 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.841835 12294 ts_tablet_manager.cc:1367] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20240502 21:29:30.842183 12297 tablet_bootstrap.cc:492] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.842303 12297 ts_tablet_manager.cc:1367] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20240502 21:29:30.842605 12294 raft_consensus.cc:348] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.842829 12294 raft_consensus.cc:374] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.842897 12297 raft_consensus.cc:348] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.843111 12297 raft_consensus.cc:374] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.843168 12297 raft_consensus.cc:729] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.843060 12295 consensus_queue.cc:260] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.843351 12297 consensus_queue.cc:260] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.843834 12297 ts_tablet_manager.cc:1398] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 21:29:30.842901 12294 raft_consensus.cc:729] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.844786 12294 consensus_queue.cc:260] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.845129 12297 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.845338 12294 ts_tablet_manager.cc:1398] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:30.846827 12294 tablet_bootstrap.cc:492] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.847551 12297 tablet_bootstrap.cc:654] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.848560 12294 tablet_bootstrap.cc:654] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.771871 12301 raft_consensus.cc:504] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.851524 12301 leader_election.cc:288] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.851903 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0b9621b33c4afcb3842c10ac4c7148" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.852249 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0b9621b33c4afcb3842c10ac4c7148" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.852610 12099 raft_consensus.cc:3037] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.854344 12295 ts_tablet_manager.cc:1398] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20240502 21:29:30.854667 12295 tablet_bootstrap.cc:492] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.855358 12099 raft_consensus.cc:2454] T 5b0b9621b33c4afcb3842c10ac4c7148 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:30.855206 11994 catalog_manager.cc:5484] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.855748 12253 raft_consensus.cc:3037] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.856277 12295 tablet_bootstrap.cc:654] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.859095 12253 raft_consensus.cc:2454] T 5b0b9621b33c4afcb3842c10ac4c7148 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:30.859511 12140 leader_election.cc:302] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.860492 12301 raft_consensus.cc:2781] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.860663 12301 raft_consensus.cc:686] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:30.861011 12301 consensus_queue.cc:237] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.861014 12297 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.862556 12297 ts_tablet_manager.cc:1367] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.001s
I20240502 21:29:30.863552 12297 raft_consensus.cc:348] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.863826 12297 raft_consensus.cc:374] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.863904 12297 raft_consensus.cc:729] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.864218 12297 consensus_queue.cc:260] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.865007 12297 ts_tablet_manager.cc:1398] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:30.865381 12297 tablet_bootstrap.cc:492] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.866643 12294 tablet_bootstrap.cc:492] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.866822 12294 ts_tablet_manager.cc:1367] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.000s
I20240502 21:29:30.866923 11996 catalog_manager.cc:5484] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.867511 12294 raft_consensus.cc:348] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.867622 12297 tablet_bootstrap.cc:654] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.867722 12294 raft_consensus.cc:374] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.867784 12294 raft_consensus.cc:729] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.867977 12301 raft_consensus.cc:482] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.868088 12301 raft_consensus.cc:504] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.869068 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3aa4f038da4e6596a28265565b2023" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.869253 12099 raft_consensus.cc:2454] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:30.869637 12140 leader_election.cc:302] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.869911 12363 raft_consensus.cc:2781] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.869994 12363 raft_consensus.cc:482] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.870051 12363 raft_consensus.cc:3037] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.872888 12363 raft_consensus.cc:504] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.873540 12363 leader_election.cc:288] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.873987 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3aa4f038da4e6596a28265565b2023" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.874174 12099 raft_consensus.cc:3037] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.873987 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3aa4f038da4e6596a28265565b2023" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.876363 12253 raft_consensus.cc:3037] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.876971 12297 tablet_bootstrap.cc:492] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.877056 12099 raft_consensus.cc:2454] T 9d3aa4f038da4e6596a28265565b2023 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:30.877234 12297 ts_tablet_manager.cc:1367] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20240502 21:29:30.877462 12140 leader_election.cc:302] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.877785 12363 raft_consensus.cc:2781] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.878283 12363 raft_consensus.cc:686] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:30.878204 12297 raft_consensus.cc:348] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.878522 12297 raft_consensus.cc:374] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.878628 12297 raft_consensus.cc:729] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.878569 12363 consensus_queue.cc:237] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.878980 12297 consensus_queue.cc:260] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.880648 12253 raft_consensus.cc:2454] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:30.881589 12294 consensus_queue.cc:260] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.882683 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3aa4f038da4e6596a28265565b2023" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.882964 12253 raft_consensus.cc:2362] T 9d3aa4f038da4e6596a28265565b2023 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1. Re-sending same reply.
I20240502 21:29:30.883512 12301 leader_election.cc:288] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.883940 12297 ts_tablet_manager.cc:1398] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20240502 21:29:30.885675 12294 ts_tablet_manager.cc:1398] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20240502 21:29:30.886085 12294 tablet_bootstrap.cc:492] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.886400 12297 tablet_bootstrap.cc:492] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.888099 12297 tablet_bootstrap.cc:654] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.888275 12294 tablet_bootstrap.cc:654] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.890535 12295 tablet_bootstrap.cc:492] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.890689 12295 ts_tablet_manager.cc:1367] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.015s
I20240502 21:29:30.891470 12295 raft_consensus.cc:348] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.891685 12295 raft_consensus.cc:374] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.891748 12295 raft_consensus.cc:729] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.891834 12335 raft_consensus.cc:482] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.891984 12335 raft_consensus.cc:504] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.891978 12295 consensus_queue.cc:260] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.892524 12295 ts_tablet_manager.cc:1398] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:30.892681 12335 leader_election.cc:288] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.892825 12295 tablet_bootstrap.cc:492] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.892463 11994 catalog_manager.cc:5484] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.893340 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5847bedd7574bd28b9eeeca3f3cf6f9" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.893396 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5847bedd7574bd28b9eeeca3f3cf6f9" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.893545 12099 raft_consensus.cc:2454] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.893589 12177 raft_consensus.cc:2454] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.893976 12219 leader_election.cc:302] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.894383 12335 raft_consensus.cc:2781] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.894466 12335 raft_consensus.cc:482] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.894527 12335 raft_consensus.cc:3037] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.894946 12295 tablet_bootstrap.cc:654] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.897838 12335 raft_consensus.cc:504] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.898801 12295 tablet_bootstrap.cc:492] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.899003 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5847bedd7574bd28b9eeeca3f3cf6f9" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.899221 12099 raft_consensus.cc:3037] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.899461 12297 tablet_bootstrap.cc:492] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.899585 12297 ts_tablet_manager.cc:1367] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.009s
I20240502 21:29:30.899823 12294 tablet_bootstrap.cc:492] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.899981 12294 ts_tablet_manager.cc:1367] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20240502 21:29:30.899009 12295 ts_tablet_manager.cc:1367] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20240502 21:29:30.900283 12297 raft_consensus.cc:348] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.900547 12297 raft_consensus.cc:374] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.900617 12297 raft_consensus.cc:729] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.900831 12294 raft_consensus.cc:348] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.900856 12297 consensus_queue.cc:260] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.901034 12294 raft_consensus.cc:374] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.901098 12294 raft_consensus.cc:729] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.901325 12294 consensus_queue.cc:260] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.901494 12295 raft_consensus.cc:348] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.901787 12295 raft_consensus.cc:374] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.901888 12295 raft_consensus.cc:729] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.902132 12295 consensus_queue.cc:260] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.902340 12297 ts_tablet_manager.cc:1398] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:30.902642 12297 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.902891 12295 ts_tablet_manager.cc:1398] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:30.903663 12295 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.902995 12294 ts_tablet_manager.cc:1398] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:30.899181 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5847bedd7574bd28b9eeeca3f3cf6f9" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.904487 12177 raft_consensus.cc:3037] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.904567 12294 tablet_bootstrap.cc:492] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.905761 12295 tablet_bootstrap.cc:654] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.903056 12335 leader_election.cc:288] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.906395 12294 tablet_bootstrap.cc:654] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.908494 12099 raft_consensus.cc:2454] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.909161 12216 leader_election.cc:302] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.909487 12335 raft_consensus.cc:2781] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.909749 12335 raft_consensus.cc:686] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:30.910015 12335 consensus_queue.cc:237] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.909759 12177 raft_consensus.cc:2454] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.911195 12297 tablet_bootstrap.cc:654] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.913405 12297 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.913458 12295 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.913550 12297 ts_tablet_manager.cc:1367] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20240502 21:29:30.913581 12295 ts_tablet_manager.cc:1367] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20240502 21:29:30.913873 11995 catalog_manager.cc:5484] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:30.914382 12297 raft_consensus.cc:348] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.914583 12297 raft_consensus.cc:374] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.914726 12297 raft_consensus.cc:729] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.914535 12295 raft_consensus.cc:348] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.914808 12295 raft_consensus.cc:374] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.914877 12295 raft_consensus.cc:729] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.914976 12297 consensus_queue.cc:260] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.915040 12295 consensus_queue.cc:260] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.915479 12295 ts_tablet_manager.cc:1398] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:30.915855 12295 tablet_bootstrap.cc:492] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.916828 12297 ts_tablet_manager.cc:1398] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:30.917124 12297 tablet_bootstrap.cc:492] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.917546 12295 tablet_bootstrap.cc:654] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.918910 12297 tablet_bootstrap.cc:654] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.921130 12294 tablet_bootstrap.cc:492] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.921309 12294 ts_tablet_manager.cc:1367] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20240502 21:29:30.921254 12316 consensus_queue.cc:1035] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:30.922339 12294 raft_consensus.cc:348] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.922614 12294 raft_consensus.cc:374] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.922694 12294 raft_consensus.cc:729] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.922966 12294 consensus_queue.cc:260] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.923612 12294 ts_tablet_manager.cc:1398] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 21:29:30.924662 12294 tablet_bootstrap.cc:492] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.927050 12294 tablet_bootstrap.cc:654] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.928545 12297 tablet_bootstrap.cc:492] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.928750 12297 ts_tablet_manager.cc:1367] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.007s
I20240502 21:29:30.929198 12316 consensus_queue.cc:1035] T 6d05591283c24d0d9e6de5ad16ab504d P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:30.930169 12297 raft_consensus.cc:348] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.930371 12295 tablet_bootstrap.cc:492] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.930430 12297 raft_consensus.cc:374] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.930537 12295 ts_tablet_manager.cc:1367] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20240502 21:29:30.931361 12295 raft_consensus.cc:348] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.931591 12295 raft_consensus.cc:374] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.931702 12295 raft_consensus.cc:729] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.931933 12295 consensus_queue.cc:260] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.932504 12295 ts_tablet_manager.cc:1398] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:30.930541 12297 raft_consensus.cc:729] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.932847 12297 consensus_queue.cc:260] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.933646 12297 ts_tablet_manager.cc:1398] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20240502 21:29:30.933771 12295 tablet_bootstrap.cc:492] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.933921 12297 tablet_bootstrap.cc:492] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.935717 12297 tablet_bootstrap.cc:654] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.936146 12335 raft_consensus.cc:482] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.936307 12335 raft_consensus.cc:504] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.936596 12295 tablet_bootstrap.cc:654] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.937043 12335 leader_election.cc:288] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.937762 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e288a51b369a41ffa0a8d2dedb849c24" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.938028 12177 raft_consensus.cc:2454] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.938400 12219 leader_election.cc:302] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.938447 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e288a51b369a41ffa0a8d2dedb849c24" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.938650 12099 raft_consensus.cc:2454] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.938702 12335 raft_consensus.cc:2781] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.938804 12335 raft_consensus.cc:482] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.938908 12335 raft_consensus.cc:3037] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.941736 12335 raft_consensus.cc:504] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.942081 12297 tablet_bootstrap.cc:492] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:30.942291 12297 ts_tablet_manager.cc:1367] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20240502 21:29:30.942597 12295 tablet_bootstrap.cc:492] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.942884 12295 ts_tablet_manager.cc:1367] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20240502 21:29:30.942888 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e288a51b369a41ffa0a8d2dedb849c24" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.943110 12335 leader_election.cc:288] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.943150 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e288a51b369a41ffa0a8d2dedb849c24" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.943177 12297 raft_consensus.cc:348] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.943382 12099 raft_consensus.cc:3037] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.943408 12297 raft_consensus.cc:374] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.943538 12297 raft_consensus.cc:729] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.943846 12295 raft_consensus.cc:348] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.944115 12295 raft_consensus.cc:374] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.944211 12295 raft_consensus.cc:729] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.944487 12294 tablet_bootstrap.cc:492] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.944512 12295 consensus_queue.cc:260] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.944650 12294 ts_tablet_manager.cc:1367] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20240502 21:29:30.945166 12295 ts_tablet_manager.cc:1398] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:30.945451 12294 raft_consensus.cc:348] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.945636 12295 tablet_bootstrap.cc:492] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.945652 12294 raft_consensus.cc:374] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.945784 12294 raft_consensus.cc:729] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.945996 12294 consensus_queue.cc:260] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.946377 12099 raft_consensus.cc:2454] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.946664 12294 ts_tablet_manager.cc:1398] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:30.946949 12216 leader_election.cc:302] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.943125 12177 raft_consensus.cc:3037] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.947343 12294 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.947571 12295 tablet_bootstrap.cc:654] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.949788 12294 tablet_bootstrap.cc:654] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.950212 12177 raft_consensus.cc:2454] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:30.945662 12297 consensus_queue.cc:260] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.951139 12332 raft_consensus.cc:2781] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.951304 12332 raft_consensus.cc:686] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:30.951624 12332 consensus_queue.cc:237] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.952837 12363 raft_consensus.cc:482] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.953052 12363 raft_consensus.cc:504] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.953456 12297 ts_tablet_manager.cc:1398] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.011s user 0.001s sys 0.002s
I20240502 21:29:30.953949 12363 leader_election.cc:288] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.954510 12297 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:30.954715 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82c29bb30eb4515ae709c0144a8998b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.955019 12100 raft_consensus.cc:2454] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:30.955636 12140 leader_election.cc:302] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.956182 12363 raft_consensus.cc:2781] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.956337 12363 raft_consensus.cc:482] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.956458 12363 raft_consensus.cc:3037] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.956383 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82c29bb30eb4515ae709c0144a8998b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.956638 12253 raft_consensus.cc:2454] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:30.961223 12297 tablet_bootstrap.cc:654] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.961728 12294 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:30.961966 12294 ts_tablet_manager.cc:1367] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20240502 21:29:30.962272 12363 raft_consensus.cc:504] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.962899 12294 raft_consensus.cc:348] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.963194 12294 raft_consensus.cc:374] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.963248 12363 leader_election.cc:288] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.963266 12294 raft_consensus.cc:729] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.963766 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82c29bb30eb4515ae709c0144a8998b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.963793 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82c29bb30eb4515ae709c0144a8998b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.964141 12295 tablet_bootstrap.cc:492] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:30.964181 12099 raft_consensus.cc:3037] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.964208 12294 consensus_queue.cc:260] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.964345 12295 ts_tablet_manager.cc:1367] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.008s
I20240502 21:29:30.964777 12294 ts_tablet_manager.cc:1398] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240502 21:29:30.965433 12295 raft_consensus.cc:348] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.965714 12295 raft_consensus.cc:374] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:30.965835 12295 raft_consensus.cc:729] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:30.966089 12309 raft_consensus.cc:482] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.966162 12295 consensus_queue.cc:260] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.966248 12309 raft_consensus.cc:504] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.966933 12295 ts_tablet_manager.cc:1398] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:30.967036 12099 raft_consensus.cc:2454] T a82c29bb30eb4515ae709c0144a8998b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:30.967327 12253 raft_consensus.cc:3037] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.967463 12140 leader_election.cc:302] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.967856 12363 raft_consensus.cc:2781] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.968039 12363 raft_consensus.cc:686] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:30.968406 12295 tablet_bootstrap.cc:492] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:30.968330 12363 consensus_queue.cc:237] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.969784 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da050ef2be604dd5af90e13e6d70c74d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:30.970031 12251 raft_consensus.cc:2454] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.970194 12253 raft_consensus.cc:2454] T a82c29bb30eb4515ae709c0144a8998b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:30.970551 12065 leader_election.cc:302] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.970810 12295 tablet_bootstrap.cc:654] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.970880 12309 leader_election.cc:288] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.971056 12309 raft_consensus.cc:2781] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.971185 12309 raft_consensus.cc:482] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.971333 12309 raft_consensus.cc:3037] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.974020 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da050ef2be604dd5af90e13e6d70c74d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.974285 12177 raft_consensus.cc:2454] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:30.975123 12335 raft_consensus.cc:482] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:30.975306 12335 raft_consensus.cc:504] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.976070 12335 leader_election.cc:288] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.977519 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c069ca9b95cc4e8ea75c71ea6cefdcc3" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:30.977598 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c069ca9b95cc4e8ea75c71ea6cefdcc3" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:30.977797 12177 raft_consensus.cc:2454] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.977972 12309 raft_consensus.cc:504] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.978251 12219 leader_election.cc:302] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:30.978566 12335 raft_consensus.cc:2781] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:30.978713 12335 raft_consensus.cc:482] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:30.978833 12335 raft_consensus.cc:3037] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.978982 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da050ef2be604dd5af90e13e6d70c74d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.979212 12177 raft_consensus.cc:3037] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.981762 12177 raft_consensus.cc:2454] T da050ef2be604dd5af90e13e6d70c74d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:30.981971 12099 raft_consensus.cc:2454] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:30.982321 12066 leader_election.cc:302] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:30.982605 12316 raft_consensus.cc:2781] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:30.981787 12335 raft_consensus.cc:504] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.982758 12316 raft_consensus.cc:686] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:30.983045 12316 consensus_queue.cc:237] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:30.983299 12335 leader_election.cc:288] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:30.983713 12309 leader_election.cc:288] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:30.984195 12294 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:30.986078 12294 tablet_bootstrap.cc:654] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:30.986675 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c069ca9b95cc4e8ea75c71ea6cefdcc3" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:30.986932 12099 raft_consensus.cc:3037] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.989120 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da050ef2be604dd5af90e13e6d70c74d" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:30.989398 12253 raft_consensus.cc:3037] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.991355 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c069ca9b95cc4e8ea75c71ea6cefdcc3" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:30.991642 12177 raft_consensus.cc:3037] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:30.992026 11993 catalog_manager.cc:5484] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:30.999573 12099 raft_consensus.cc:2454] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.000126 12216 leader_election.cc:302] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.000244 12294 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.000408 12294 ts_tablet_manager.cc:1367] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20240502 21:29:31.000528 12335 raft_consensus.cc:2781] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.000691 12335 raft_consensus.cc:686] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.001021 12335 consensus_queue.cc:237] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.001590 12294 raft_consensus.cc:348] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.001821 12294 raft_consensus.cc:374] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.001941 12294 raft_consensus.cc:729] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.002182 12294 consensus_queue.cc:260] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.002898 12294 ts_tablet_manager.cc:1398] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:31.003305 12294 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.004585 12364 raft_consensus.cc:482] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.004761 12364 raft_consensus.cc:504] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.005514 12364 leader_election.cc:288] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.006049 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bec182656a4200b4e9938e861ab21f" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.006394 12176 raft_consensus.cc:2379] T 59bec182656a4200b4e9938e861ab21f P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0672163bb07b41a1b14342a8efcb24a0 in current term 1: Already voted for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in this term.
I20240502 21:29:31.006829 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59bec182656a4200b4e9938e861ab21f" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.007835 12065 leader_election.cc:302] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.008420 12364 raft_consensus.cc:3037] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.011874 12364 raft_consensus.cc:2726] T 59bec182656a4200b4e9938e861ab21f P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.012241 12294 tablet_bootstrap.cc:654] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.014302 12253 raft_consensus.cc:2454] T da050ef2be604dd5af90e13e6d70c74d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:31.015213 12177 raft_consensus.cc:2454] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.021975 12294 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.022255 12294 ts_tablet_manager.cc:1367] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.000s
I20240502 21:29:31.023403 12294 raft_consensus.cc:348] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.023764 12294 raft_consensus.cc:374] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.023875 12294 raft_consensus.cc:729] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.024319 12294 consensus_queue.cc:260] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.025573 12294 ts_tablet_manager.cc:1398] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240502 21:29:31.026145 12294 tablet_bootstrap.cc:492] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.026619 12363 raft_consensus.cc:482] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.026860 12363 raft_consensus.cc:504] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.027702 12363 leader_election.cc:288] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.028668 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe982f82c003411e9f9dda42378399f8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.028923 12099 raft_consensus.cc:2454] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.029397 12140 leader_election.cc:302] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.029765 12363 raft_consensus.cc:2781] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.029882 12363 raft_consensus.cc:482] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.030043 12363 raft_consensus.cc:3037] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.030861 12294 tablet_bootstrap.cc:654] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.035346 12363 raft_consensus.cc:504] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.035476 12364 raft_consensus.cc:482] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.035682 12364 raft_consensus.cc:504] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.036252 12364 leader_election.cc:288] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.036409 12363 leader_election.cc:288] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.036767 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe982f82c003411e9f9dda42378399f8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.036870 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe982f82c003411e9f9dda42378399f8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.037040 12253 raft_consensus.cc:2454] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.038522 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.039012 12065 leader_election.cc:341] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.039433 12364 raft_consensus.cc:482] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.039579 12364 raft_consensus.cc:504] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.039403 11993 catalog_manager.cc:5484] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.040223 12364 leader_election.cc:288] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.041059 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09266cc9ff6c453bbf4cd0a1621375db" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.041292 12176 raft_consensus.cc:2454] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:31.041713 12066 leader_election.cc:302] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.042011 12364 raft_consensus.cc:2781] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.042145 12364 raft_consensus.cc:482] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.042254 12364 raft_consensus.cc:3037] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.043392 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.043833 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe982f82c003411e9f9dda42378399f8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.044049 12099 raft_consensus.cc:3037] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:31.045428 12066 leader_election.cc:341] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.045584 12066 leader_election.cc:302] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.047158 12099 raft_consensus.cc:2454] T fe982f82c003411e9f9dda42378399f8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.047598 12140 leader_election.cc:302] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.048096 12363 raft_consensus.cc:2781] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.048808 12309 raft_consensus.cc:2726] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.049034 12309 raft_consensus.cc:482] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.049211 12309 raft_consensus.cc:504] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.049654 12364 raft_consensus.cc:504] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.049945 12309 leader_election.cc:288] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.050611 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a147afb591461387bfe6ed7587a6ba" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.050835 12251 raft_consensus.cc:2454] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:31.051489 12065 leader_election.cc:302] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.051862 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09266cc9ff6c453bbf4cd0a1621375db" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.052138 12176 raft_consensus.cc:3037] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.052228 12364 leader_election.cc:288] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.052428 12364 raft_consensus.cc:2781] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.052541 12364 raft_consensus.cc:482] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.052664 12364 raft_consensus.cc:3037] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.055272 12176 raft_consensus.cc:2454] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:31.055744 12066 leader_election.cc:302] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.055910 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09266cc9ff6c453bbf4cd0a1621375db" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.056229 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09266cc9ff6c453bbf4cd0a1621375db" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.056478 12309 raft_consensus.cc:2781] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.055272 12364 raft_consensus.cc:504] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.056648 12309 raft_consensus.cc:686] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:31.056972 12309 consensus_queue.cc:237] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.057512 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a147afb591461387bfe6ed7587a6ba" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
W20240502 21:29:31.057966 12066 leader_election.cc:341] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.058400 12361 raft_consensus.cc:482] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.058616 12361 raft_consensus.cc:504] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.059551 12361 leader_election.cc:288] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:31.060202 12065 leader_election.cc:341] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.060202 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6b3c759a8b4236b3712d5161bdb8dc" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.060459 12100 raft_consensus.cc:2454] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
W20240502 21:29:31.060634 12065 leader_election.cc:341] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.060896 12140 leader_election.cc:302] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.060940 12367 raft_consensus.cc:482] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.061303 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a147afb591461387bfe6ed7587a6ba" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.061245 12367 raft_consensus.cc:504] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.061522 12251 raft_consensus.cc:3037] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.062008 12367 leader_election.cc:288] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.062206 12361 raft_consensus.cc:2781] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.062327 12361 raft_consensus.cc:482] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.062434 12361 raft_consensus.cc:3037] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.065168 12361 raft_consensus.cc:504] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.065425 12363 raft_consensus.cc:686] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.065722 12363 consensus_queue.cc:237] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.066385 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6b3c759a8b4236b3712d5161bdb8dc" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.066582 12100 raft_consensus.cc:3037] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.051862 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a147afb591461387bfe6ed7587a6ba" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.070354 12100 raft_consensus.cc:2454] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.071151 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c0955962874018b574931aea7bd0ce" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.071719 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6b3c759a8b4236b3712d5161bdb8dc" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
W20240502 21:29:31.072293 12142 leader_election.cc:341] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.072533 12361 leader_election.cc:288] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:31.072963 12066 leader_election.cc:341] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:31.073331 12066 leader_election.cc:341] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.073592 12140 leader_election.cc:302] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.074510 12308 consensus_queue.cc:1035] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.075145 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c0955962874018b574931aea7bd0ce" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.075978 12065 leader_election.cc:341] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.076164 12065 leader_election.cc:302] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.076297 12364 leader_election.cc:288] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.065300 12251 raft_consensus.cc:2454] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:31.076589 12364 raft_consensus.cc:2726] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.076897 12065 leader_election.cc:302] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.078060 12361 raft_consensus.cc:2781] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.078253 12361 raft_consensus.cc:686] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.078629 12361 consensus_queue.cc:237] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.079495 12364 raft_consensus.cc:2781] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.079802 12364 raft_consensus.cc:686] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:31.079926 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6b3c759a8b4236b3712d5161bdb8dc" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.080144 12364 consensus_queue.cc:237] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
W20240502 21:29:31.080615 12142 leader_election.cc:341] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.092741 11993 catalog_manager.cc:5484] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.098441 12295 tablet_bootstrap.cc:492] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.098659 12295 ts_tablet_manager.cc:1367] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.130s user 0.046s sys 0.078s
I20240502 21:29:31.099344 11995 catalog_manager.cc:5484] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.100528 11996 catalog_manager.cc:5484] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.102453 12335 consensus_queue.cc:1035] T 59bec182656a4200b4e9938e861ab21f P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.099494 12295 raft_consensus.cc:348] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.103160 12295 raft_consensus.cc:374] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.103260 12295 raft_consensus.cc:729] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.103621 12294 tablet_bootstrap.cc:492] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.103564 12295 consensus_queue.cc:260] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.103806 12294 ts_tablet_manager.cc:1367] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.078s user 0.009s sys 0.001s
I20240502 21:29:31.103698 11993 catalog_manager.cc:5484] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.104310 12295 ts_tablet_manager.cc:1398] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20240502 21:29:31.104666 12295 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.104732 12294 raft_consensus.cc:348] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.104946 12294 raft_consensus.cc:374] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.105052 12294 raft_consensus.cc:729] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.105273 12294 consensus_queue.cc:260] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.105875 12294 ts_tablet_manager.cc:1398] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:31.106181 12294 tablet_bootstrap.cc:492] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.106395 12295 tablet_bootstrap.cc:654] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.108714 12294 tablet_bootstrap.cc:654] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.114624 12295 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.114832 12295 ts_tablet_manager.cc:1367] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20240502 21:29:31.115633 12295 raft_consensus.cc:348] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.115868 12295 raft_consensus.cc:374] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.115999 12295 raft_consensus.cc:729] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.116335 12295 consensus_queue.cc:260] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.116667 11994 catalog_manager.cc:5484] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.116956 12295 ts_tablet_manager.cc:1398] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20240502 21:29:31.117431 12295 tablet_bootstrap.cc:492] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.117516 12294 tablet_bootstrap.cc:492] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.119448 12294 ts_tablet_manager.cc:1367] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20240502 21:29:31.119746 12295 tablet_bootstrap.cc:654] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.120074 12297 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.120252 12297 ts_tablet_manager.cc:1367] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.166s user 0.063s sys 0.063s
I20240502 21:29:31.120347 12294 raft_consensus.cc:348] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.120594 12294 raft_consensus.cc:374] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.120685 12294 raft_consensus.cc:729] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.120973 12294 consensus_queue.cc:260] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.121049 12297 raft_consensus.cc:348] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.121248 12297 raft_consensus.cc:374] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.121361 12297 raft_consensus.cc:729] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.121624 12297 consensus_queue.cc:260] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.122189 12297 ts_tablet_manager.cc:1398] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.122530 12297 tablet_bootstrap.cc:492] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.124475 12297 tablet_bootstrap.cc:654] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.124540 12294 ts_tablet_manager.cc:1398] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:29:31.127768 12294 tablet_bootstrap.cc:492] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.130642 12294 tablet_bootstrap.cc:654] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.131768 12335 raft_consensus.cc:482] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.132871 12335 raft_consensus.cc:504] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.133616 12297 tablet_bootstrap.cc:492] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.133894 12297 ts_tablet_manager.cc:1367] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20240502 21:29:31.134485 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d88ae6514545bca16f20beb52b8e7b" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.135015 12297 raft_consensus.cc:348] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.135339 12297 raft_consensus.cc:374] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.135445 12297 raft_consensus.cc:729] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.135857 12297 consensus_queue.cc:260] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.136345 12364 consensus_queue.cc:1035] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.136868 12297 ts_tablet_manager.cc:1398] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 21:29:31.137372 12297 tablet_bootstrap.cc:492] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.137315 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d88ae6514545bca16f20beb52b8e7b" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.137676 12176 raft_consensus.cc:2379] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:31.139204 12219 leader_election.cc:302] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.139710 12332 raft_consensus.cc:3037] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.140018 12297 tablet_bootstrap.cc:654] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.140372 12335 leader_election.cc:288] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.140842 12335 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.142752 12332 raft_consensus.cc:2726] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.146981 12294 tablet_bootstrap.cc:492] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.146987 12364 raft_consensus.cc:482] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.147217 12294 ts_tablet_manager.cc:1367] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.000s
I20240502 21:29:31.147214 12364 raft_consensus.cc:504] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.148655 12364 leader_election.cc:288] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.149652 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa48c97f4426444585651368f28b265a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.150277 12065 leader_election.cc:341] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.151399 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa48c97f4426444585651368f28b265a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.151587 12308 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 21:29:31.152208 12066 leader_election.cc:341] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.152315 12066 leader_election.cc:302] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.153105 12294 raft_consensus.cc:348] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.153297 12294 raft_consensus.cc:374] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.153344 12294 raft_consensus.cc:729] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.153528 12309 raft_consensus.cc:2726] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.153625 12294 consensus_queue.cc:260] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.154198 12294 ts_tablet_manager.cc:1398] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20240502 21:29:31.154516 12294 tablet_bootstrap.cc:492] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.156231 12294 tablet_bootstrap.cc:654] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.157439 12364 consensus_queue.cc:1035] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.158425 11993 catalog_manager.cc:5484] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.166810 12294 tablet_bootstrap.cc:492] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.169188 12294 ts_tablet_manager.cc:1367] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.004s
I20240502 21:29:31.167744 12297 tablet_bootstrap.cc:492] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.169885 12297 ts_tablet_manager.cc:1367] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.011s
I20240502 21:29:31.169982 12294 raft_consensus.cc:348] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.170183 12294 raft_consensus.cc:374] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.170235 12294 raft_consensus.cc:729] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.170464 12294 consensus_queue.cc:260] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.170539 12297 raft_consensus.cc:348] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.170713 12297 raft_consensus.cc:374] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.170770 12297 raft_consensus.cc:729] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.171056 12294 ts_tablet_manager.cc:1398] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:31.171000 12297 consensus_queue.cc:260] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.171326 12294 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.171607 12297 ts_tablet_manager.cc:1398] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.172708 12294 tablet_bootstrap.cc:654] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.174224 12295 tablet_bootstrap.cc:492] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.174360 12295 ts_tablet_manager.cc:1367] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.057s user 0.012s sys 0.032s
I20240502 21:29:31.175042 12295 raft_consensus.cc:348] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.175225 12295 raft_consensus.cc:374] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.175280 12295 raft_consensus.cc:729] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.175482 12295 consensus_queue.cc:260] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.176090 12295 ts_tablet_manager.cc:1398] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:31.176384 12295 tablet_bootstrap.cc:492] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.177790 12295 tablet_bootstrap.cc:654] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.179543 12294 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.179543 12295 tablet_bootstrap.cc:492] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.179708 12295 ts_tablet_manager.cc:1367] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20240502 21:29:31.179708 12294 ts_tablet_manager.cc:1367] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20240502 21:29:31.180413 12295 raft_consensus.cc:348] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.180413 12294 raft_consensus.cc:348] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.180663 12295 raft_consensus.cc:374] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.180672 12294 raft_consensus.cc:374] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.180737 12295 raft_consensus.cc:729] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.180742 12294 raft_consensus.cc:729] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.180965 12294 consensus_queue.cc:260] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.180965 12295 consensus_queue.cc:260] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.181645 12295 ts_tablet_manager.cc:1398] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:31.181910 12295 tablet_bootstrap.cc:492] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.181936 12297 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.183264 12297 tablet_bootstrap.cc:654] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.183311 12295 tablet_bootstrap.cc:654] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.185036 12294 ts_tablet_manager.cc:1398] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 21:29:31.185410 12294 tablet_bootstrap.cc:492] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.187481 12294 tablet_bootstrap.cc:654] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.190687 12294 tablet_bootstrap.cc:492] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.190889 12294 ts_tablet_manager.cc:1367] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20240502 21:29:31.191716 12294 raft_consensus.cc:348] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.191910 12294 raft_consensus.cc:374] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.192005 12294 raft_consensus.cc:729] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.192251 12294 consensus_queue.cc:260] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.192703 12294 ts_tablet_manager.cc:1398] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:31.193037 12294 tablet_bootstrap.cc:492] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.195210 12363 raft_consensus.cc:482] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.195349 12363 raft_consensus.cc:504] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.196038 12363 leader_election.cc:288] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.196383 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244c153efb7943f48ddee849bb0187c4" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.196543 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244c153efb7943f48ddee849bb0187c4" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.196606 12100 raft_consensus.cc:2454] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
W20240502 21:29:31.197023 12142 leader_election.cc:341] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.197206 12140 leader_election.cc:302] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.197422 12363 raft_consensus.cc:2781] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.197531 12363 raft_consensus.cc:482] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.197595 12363 raft_consensus.cc:3037] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.204471 12335 raft_consensus.cc:482] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.204634 12335 raft_consensus.cc:504] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.205263 12335 leader_election.cc:288] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.205662 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434ffa8e08cf45f6905eb6aaaab820ca" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.205770 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434ffa8e08cf45f6905eb6aaaab820ca" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.205909 12175 raft_consensus.cc:2454] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.205873 12100 raft_consensus.cc:2454] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.206249 12216 leader_election.cc:302] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.206243 12364 consensus_queue.cc:1035] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.206599 12335 raft_consensus.cc:2781] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.206684 12335 raft_consensus.cc:482] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.206758 12335 raft_consensus.cc:3037] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.208169 12294 tablet_bootstrap.cc:654] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.208580 12297 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.208735 12297 ts_tablet_manager.cc:1367] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.002s
I20240502 21:29:31.209581 12297 raft_consensus.cc:348] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.209774 12297 raft_consensus.cc:374] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.209820 12297 raft_consensus.cc:729] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.210012 12297 consensus_queue.cc:260] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.210573 12297 ts_tablet_manager.cc:1398] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.210623 12335 raft_consensus.cc:504] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.210871 12297 tablet_bootstrap.cc:492] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.211254 12335 leader_election.cc:288] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.211463 12363 raft_consensus.cc:504] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.211854 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434ffa8e08cf45f6905eb6aaaab820ca" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.211860 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434ffa8e08cf45f6905eb6aaaab820ca" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.212083 12177 raft_consensus.cc:3037] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.212148 12100 raft_consensus.cc:3037] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.212571 12297 tablet_bootstrap.cc:654] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.214694 12177 raft_consensus.cc:2454] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.215277 12100 raft_consensus.cc:2454] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.215752 12216 leader_election.cc:302] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.215902 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244c153efb7943f48ddee849bb0187c4" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.216118 12099 raft_consensus.cc:3037] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.216405 12363 leader_election.cc:288] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.217821 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "244c153efb7943f48ddee849bb0187c4" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.218011 12253 raft_consensus.cc:3037] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:31.219252 12046 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240502 21:29:31.219760 12099 raft_consensus.cc:2454] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.220593 12294 tablet_bootstrap.cc:492] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.220724 12253 raft_consensus.cc:2454] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.220865 12294 ts_tablet_manager.cc:1367] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.003s
I20240502 21:29:31.221376 12142 leader_election.cc:302] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.221727 12294 raft_consensus.cc:348] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.221856 12363 raft_consensus.cc:2781] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.221977 12294 raft_consensus.cc:374] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.222092 12294 raft_consensus.cc:729] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.216121 12335 raft_consensus.cc:2781] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.222362 12294 consensus_queue.cc:260] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.222769 12335 raft_consensus.cc:686] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.223562 12294 ts_tablet_manager.cc:1398] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.223929 12294 tablet_bootstrap.cc:492] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.224601 12332 raft_consensus.cc:482] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.224789 12332 raft_consensus.cc:504] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.225309 12364 consensus_queue.cc:1035] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.225612 12332 leader_election.cc:288] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.225898 12294 tablet_bootstrap.cc:654] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.228086 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4afa7764f554f62abb539fe13838764" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.228768 12219 leader_election.cc:341] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.229202 12335 consensus_queue.cc:237] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.229996 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4afa7764f554f62abb539fe13838764" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.230247 12099 raft_consensus.cc:2454] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.232378 12216 leader_election.cc:302] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.232671 12363 raft_consensus.cc:686] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.232998 12332 raft_consensus.cc:2781] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.233151 12332 raft_consensus.cc:482] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.233309 12332 raft_consensus.cc:3037] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.233091 12363 consensus_queue.cc:237] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.237345 12294 tablet_bootstrap.cc:492] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.237437 12332 raft_consensus.cc:504] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.237778 12294 ts_tablet_manager.cc:1367] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20240502 21:29:31.238886 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4afa7764f554f62abb539fe13838764" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.238890 12294 raft_consensus.cc:348] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.239288 12098 raft_consensus.cc:3037] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.239472 12294 raft_consensus.cc:374] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.239660 12294 raft_consensus.cc:729] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.240093 12294 consensus_queue.cc:260] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.240948 12294 ts_tablet_manager.cc:1398] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.241459 12294 tablet_bootstrap.cc:492] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.243124 12294 tablet_bootstrap.cc:654] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.244489 12332 leader_election.cc:288] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.245172 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4afa7764f554f62abb539fe13838764" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.248303 12295 tablet_bootstrap.cc:492] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
W20240502 21:29:31.248562 12219 leader_election.cc:341] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.249166 12295 ts_tablet_manager.cc:1367] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.003s
I20240502 21:29:31.249532 11994 catalog_manager.cc:5484] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.251305 12295 raft_consensus.cc:348] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.251740 12295 raft_consensus.cc:374] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.251905 12295 raft_consensus.cc:729] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.252357 12295 consensus_queue.cc:260] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.253484 12295 ts_tablet_manager.cc:1398] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.253991 12295 tablet_bootstrap.cc:492] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.249038 12098 raft_consensus.cc:2454] T a4afa7764f554f62abb539fe13838764 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.256611 12216 leader_election.cc:302] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.257223 12332 raft_consensus.cc:2781] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.257457 12332 raft_consensus.cc:686] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.257848 12332 consensus_queue.cc:237] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.256719 12297 tablet_bootstrap.cc:492] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.258461 12297 ts_tablet_manager.cc:1367] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.048s user 0.029s sys 0.011s
I20240502 21:29:31.259495 12297 raft_consensus.cc:348] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.260216 12297 raft_consensus.cc:374] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.260394 12297 raft_consensus.cc:729] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.260077 12295 tablet_bootstrap.cc:654] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.261891 12297 consensus_queue.cc:260] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.262807 12297 ts_tablet_manager.cc:1398] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:31.263335 12297 tablet_bootstrap.cc:492] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.265664 12297 tablet_bootstrap.cc:654] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.267809 12294 tablet_bootstrap.cc:492] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.268071 12294 ts_tablet_manager.cc:1367] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.013s
I20240502 21:29:31.268852 12294 raft_consensus.cc:348] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.269097 12294 raft_consensus.cc:374] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.269215 12294 raft_consensus.cc:729] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.269270 12297 tablet_bootstrap.cc:492] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.269297 12295 tablet_bootstrap.cc:492] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.269456 12297 ts_tablet_manager.cc:1367] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240502 21:29:31.269460 12295 ts_tablet_manager.cc:1367] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20240502 21:29:31.269532 12294 consensus_queue.cc:260] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.269706 11994 catalog_manager.cc:5484] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.270262 12295 raft_consensus.cc:348] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.270517 12295 raft_consensus.cc:374] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.270656 12295 raft_consensus.cc:729] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.270939 12295 consensus_queue.cc:260] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.271077 12294 ts_tablet_manager.cc:1398] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.271548 12295 ts_tablet_manager.cc:1398] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.270536 12297 raft_consensus.cc:348] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.271770 12297 raft_consensus.cc:374] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.272081 12297 raft_consensus.cc:729] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.272419 12297 consensus_queue.cc:260] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.271863 12295 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.271982 12294 tablet_bootstrap.cc:492] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.273036 12297 ts_tablet_manager.cc:1398] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.274659 12295 tablet_bootstrap.cc:654] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.274971 12297 tablet_bootstrap.cc:492] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.275588 12294 tablet_bootstrap.cc:654] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.276682 12297 tablet_bootstrap.cc:654] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.278026 12363 raft_consensus.cc:482] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.278219 12363 raft_consensus.cc:504] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.278924 12363 leader_election.cc:288] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.279500 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db39150ee1094ace88cbc8a2c105985a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.279762 12098 raft_consensus.cc:2454] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.280207 12140 leader_election.cc:302] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.280375 12294 tablet_bootstrap.cc:492] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.280563 12294 ts_tablet_manager.cc:1367] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20240502 21:29:31.280870 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db39150ee1094ace88cbc8a2c105985a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.281358 12142 leader_election.cc:341] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.281308 12294 raft_consensus.cc:348] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.281560 12294 raft_consensus.cc:374] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.281653 12294 raft_consensus.cc:729] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.281785 12361 raft_consensus.cc:2781] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.281925 12361 raft_consensus.cc:482] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.282049 12361 raft_consensus.cc:3037] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.281935 12294 consensus_queue.cc:260] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.282644 12294 ts_tablet_manager.cc:1398] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.282963 12294 tablet_bootstrap.cc:492] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.284523 12294 tablet_bootstrap.cc:654] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.285837 11993 catalog_manager.cc:5484] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.290416 12363 raft_consensus.cc:482] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.290654 12363 raft_consensus.cc:504] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.291679 12363 leader_election.cc:288] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.292819 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.293049 12099 raft_consensus.cc:2454] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.293479 12335 consensus_queue.cc:1035] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.293956 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.293474 12140 leader_election.cc:302] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
W20240502 21:29:31.296772 12142 leader_election.cc:341] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.297241 12301 raft_consensus.cc:2781] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.297425 12301 raft_consensus.cc:482] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.297564 12301 raft_consensus.cc:3037] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.300750 12335 raft_consensus.cc:482] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.301002 12335 raft_consensus.cc:504] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.302182 12335 leader_election.cc:288] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.303118 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0766fbf50cb4b17a55ee3a21de8248c" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.303484 12098 raft_consensus.cc:2454] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.304131 12216 leader_election.cc:302] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.304782 12335 raft_consensus.cc:2781] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.304951 12335 raft_consensus.cc:482] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.305090 12335 raft_consensus.cc:3037] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.308712 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0766fbf50cb4b17a55ee3a21de8248c" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.309072 12175 raft_consensus.cc:2454] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.311342 12361 raft_consensus.cc:504] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.312810 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db39150ee1094ace88cbc8a2c105985a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.313122 12098 raft_consensus.cc:3037] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.318912 12363 raft_consensus.cc:482] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.319185 12363 raft_consensus.cc:504] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.320183 12363 leader_election.cc:288] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.321014 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5d3606b31841f6986c1c0954eb9332" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.321357 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5d3606b31841f6986c1c0954eb9332" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.321827 12100 raft_consensus.cc:2454] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.322502 12140 leader_election.cc:302] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.324717 12301 raft_consensus.cc:504] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.326140 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.326467 12100 raft_consensus.cc:3037] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:31.330473 12142 leader_election.cc:341] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.334252 12098 raft_consensus.cc:2454] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.334941 12140 leader_election.cc:302] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.335850 12363 raft_consensus.cc:2781] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.335994 12361 leader_election.cc:288] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.336881 12332 raft_consensus.cc:482] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.336957 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db39150ee1094ace88cbc8a2c105985a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.337107 12332 raft_consensus.cc:504] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
W20240502 21:29:31.337695 12142 leader_election.cc:341] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.338054 12332 leader_election.cc:288] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.338723 12100 raft_consensus.cc:2454] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.338864 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9bd1fc947f428197d8747df293dffd" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.339155 12097 raft_consensus.cc:2454] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.339238 12140 leader_election.cc:302] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.339699 12216 leader_election.cc:302] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.340095 12332 raft_consensus.cc:2781] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.340260 12332 raft_consensus.cc:482] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.340302 12361 raft_consensus.cc:2781] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.340399 12332 raft_consensus.cc:3037] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.340497 12361 raft_consensus.cc:686] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.340909 12361 consensus_queue.cc:237] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.343021 12301 leader_election.cc:288] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.343950 12448 raft_consensus.cc:2781] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.344118 12448 raft_consensus.cc:482] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.344259 12448 raft_consensus.cc:3037] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.342176 12335 raft_consensus.cc:504] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.344974 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
W20240502 21:29:31.345746 12142 leader_election.cc:341] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.346877 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0766fbf50cb4b17a55ee3a21de8248c" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.347133 12098 raft_consensus.cc:3037] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.348206 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9bd1fc947f428197d8747df293dffd" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.349083 12219 leader_election.cc:341] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.349668 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0766fbf50cb4b17a55ee3a21de8248c" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.349910 12175 raft_consensus.cc:3037] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.351065 12363 raft_consensus.cc:686] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.351363 12098 raft_consensus.cc:2454] T b0766fbf50cb4b17a55ee3a21de8248c P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.351456 12363 consensus_queue.cc:237] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.351836 12216 leader_election.cc:302] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.352191 12334 raft_consensus.cc:2781] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.352349 12334 raft_consensus.cc:686] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.352699 12334 consensus_queue.cc:237] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.353607 12175 raft_consensus.cc:2454] T b0766fbf50cb4b17a55ee3a21de8248c P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.347241 12332 raft_consensus.cc:504] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.354528 12448 raft_consensus.cc:504] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.355054 12308 raft_consensus.cc:482] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.355244 12308 raft_consensus.cc:504] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.355334 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9bd1fc947f428197d8747df293dffd" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.355573 12100 raft_consensus.cc:3037] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.356026 12308 leader_election.cc:288] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.356926 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09266cc9ff6c453bbf4cd0a1621375db" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.358333 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5d3606b31841f6986c1c0954eb9332" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.358619 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5d3606b31841f6986c1c0954eb9332" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.358886 12097 raft_consensus.cc:3037] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.359066 12332 leader_election.cc:288] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.359370 12448 leader_election.cc:288] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.360862 12100 raft_consensus.cc:2454] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.362769 12097 raft_consensus.cc:2454] T 9e5d3606b31841f6986c1c0954eb9332 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.363510 12216 leader_election.cc:302] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.363903 12308 raft_consensus.cc:2781] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.365473 12332 raft_consensus.cc:482] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.365676 12332 raft_consensus.cc:504] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.367127 12332 leader_election.cc:288] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.367502 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c0955962874018b574931aea7bd0ce" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.367764 12097 raft_consensus.cc:2454] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.368299 12216 leader_election.cc:302] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.368640 12140 leader_election.cc:302] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.368695 12332 raft_consensus.cc:2781] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.368824 12332 raft_consensus.cc:482] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.368954 12332 raft_consensus.cc:3037] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.368958 12449 raft_consensus.cc:2781] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.369307 12449 raft_consensus.cc:686] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.369719 12449 consensus_queue.cc:237] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.372784 12332 raft_consensus.cc:504] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.374126 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c0955962874018b574931aea7bd0ce" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.374399 12096 raft_consensus.cc:3037] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:31.374612 12142 leader_election.cc:341] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.375535 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9bd1fc947f428197d8747df293dffd" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.376200 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09266cc9ff6c453bbf4cd0a1621375db" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.376577 12175 raft_consensus.cc:2379] T 09266cc9ff6c453bbf4cd0a1621375db P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:31.377573 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c0955962874018b574931aea7bd0ce" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.377822 12177 raft_consensus.cc:2454] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.378295 12096 raft_consensus.cc:2454] T d6c0955962874018b574931aea7bd0ce P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
W20240502 21:29:31.378756 12219 leader_election.cc:341] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.379261 12219 leader_election.cc:302] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.380337 12216 leader_election.cc:302] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.381050 12455 raft_consensus.cc:3037] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.382447 12316 raft_consensus.cc:482] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.382666 12316 raft_consensus.cc:504] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.383608 12316 leader_election.cc:288] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.384352 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c958a20984244369aeed6a8442d51dd" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.384887 12065 leader_election.cc:341] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.386674 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c958a20984244369aeed6a8442d51dd" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.387583 12066 leader_election.cc:341] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.387761 12066 leader_election.cc:302] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.388316 12316 raft_consensus.cc:2726] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.388904 12455 raft_consensus.cc:2726] T 09266cc9ff6c453bbf4cd0a1621375db P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.388954 12332 leader_election.cc:288] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.389586 12455 raft_consensus.cc:2781] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.389998 12455 raft_consensus.cc:686] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.389546 12332 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.390765 12455 consensus_queue.cc:237] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.391386 12448 consensus_queue.cc:1035] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.392091 12335 leader_election.cc:288] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.393724 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c0955962874018b574931aea7bd0ce" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.394011 12177 raft_consensus.cc:3037] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.397670 12177 raft_consensus.cc:2454] T d6c0955962874018b574931aea7bd0ce P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.409770 12316 raft_consensus.cc:482] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.410100 12316 raft_consensus.cc:504] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.411713 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834684d85d644236aab5229b549c428a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.411789 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834684d85d644236aab5229b549c428a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.411988 12316 leader_election.cc:288] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:31.412586 12065 leader_election.cc:341] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:31.413043 12066 leader_election.cc:341] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.413239 12066 leader_election.cc:302] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.413581 12316 raft_consensus.cc:2726] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.418016 12316 raft_consensus.cc:482] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.418246 12316 raft_consensus.cc:504] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.419574 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.420027 12316 leader_election.cc:288] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:31.420706 12065 leader_election.cc:341] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.421669 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.422309 12066 leader_election.cc:341] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.422484 12066 leader_election.cc:302] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.423004 12316 raft_consensus.cc:2726] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.423332 12335 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.425272 12366 raft_consensus.cc:482] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.425516 12366 raft_consensus.cc:504] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.426369 12366 leader_election.cc:288] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.426731 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aef08a96a4e49f6bc1d3fc3175d1685" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.426963 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aef08a96a4e49f6bc1d3fc3175d1685" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.427282 12066 leader_election.cc:341] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:31.427464 12065 leader_election.cc:341] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.427619 12065 leader_election.cc:302] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.428187 12366 raft_consensus.cc:2726] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.440124 12308 raft_consensus.cc:686] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.440737 12308 consensus_queue.cc:237] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.442414 12361 consensus_queue.cc:1035] T 9d3aa4f038da4e6596a28265565b2023 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.445444 12454 consensus_queue.cc:1035] T b5847bedd7574bd28b9eeeca3f3cf6f9 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.447621 12361 consensus_queue.cc:1035] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.453856 12301 consensus_queue.cc:1035] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.457443 12361 consensus_queue.cc:1035] T 5b0b9621b33c4afcb3842c10ac4c7148 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.460469 12366 consensus_queue.cc:1035] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.474526 12366 raft_consensus.cc:482] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.474740 12366 raft_consensus.cc:504] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.475649 12366 leader_election.cc:288] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.476850 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.477296 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.477968 12066 leader_election.cc:341] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:31.478435 12065 leader_election.cc:341] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.478588 12065 leader_election.cc:302] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.479126 12366 raft_consensus.cc:2726] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.473809 11993 catalog_manager.cc:5484] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.480717 12454 consensus_queue.cc:1035] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.483530 12301 consensus_queue.cc:1035] T a82c29bb30eb4515ae709c0144a8998b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.488296 12335 consensus_queue.cc:1035] T c069ca9b95cc4e8ea75c71ea6cefdcc3 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.492131 12366 consensus_queue.cc:1035] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.498579 11993 catalog_manager.cc:5484] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.501621 12366 consensus_queue.cc:1035] T 09266cc9ff6c453bbf4cd0a1621375db P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.504846 12335 raft_consensus.cc:482] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.505384 12335 raft_consensus.cc:504] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.507071 12335 leader_election.cc:288] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.507119 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6b3c759a8b4236b3712d5161bdb8dc" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.511209 12366 consensus_queue.cc:1035] T da050ef2be604dd5af90e13e6d70c74d P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.512589 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6b3c759a8b4236b3712d5161bdb8dc" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.512912 12098 raft_consensus.cc:2379] T 3f6b3c759a8b4236b3712d5161bdb8dc P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 17f04379252d4ecfab3a4916c61a984f in this term.
I20240502 21:29:31.513509 12216 leader_election.cc:302] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.514089 12335 raft_consensus.cc:3037] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.517992 12335 raft_consensus.cc:2726] T 3f6b3c759a8b4236b3712d5161bdb8dc P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.528401 12295 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.528650 12295 ts_tablet_manager.cc:1367] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.257s user 0.039s sys 0.059s
I20240502 21:29:31.529479 12295 raft_consensus.cc:348] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.529767 12295 raft_consensus.cc:374] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.529875 12295 raft_consensus.cc:729] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.530187 12295 consensus_queue.cc:260] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.530964 12295 ts_tablet_manager.cc:1398] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:31.531800 12295 tablet_bootstrap.cc:492] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.534379 11993 catalog_manager.cc:5484] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.534819 12295 tablet_bootstrap.cc:654] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.535130 11993 catalog_manager.cc:5484] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.537382 12294 tablet_bootstrap.cc:492] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.537569 12294 ts_tablet_manager.cc:1367] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.255s user 0.041s sys 0.055s
I20240502 21:29:31.538367 12294 raft_consensus.cc:348] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.538590 12294 raft_consensus.cc:374] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.538697 12294 raft_consensus.cc:729] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.538944 12294 consensus_queue.cc:260] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.539544 12294 ts_tablet_manager.cc:1398] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:31.539865 12294 tablet_bootstrap.cc:492] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.541183 12297 tablet_bootstrap.cc:492] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.541375 12297 ts_tablet_manager.cc:1367] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.266s user 0.063s sys 0.162s
I20240502 21:29:31.541625 12294 tablet_bootstrap.cc:654] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.542063 12297 raft_consensus.cc:348] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.542290 12297 raft_consensus.cc:374] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.542382 12297 raft_consensus.cc:729] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.542658 12297 consensus_queue.cc:260] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.543205 12297 ts_tablet_manager.cc:1398] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.543542 12297 tablet_bootstrap.cc:492] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.544275 11995 catalog_manager.cc:5484] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.544881 11995 catalog_manager.cc:5484] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.545075 12297 tablet_bootstrap.cc:654] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.546439 12294 tablet_bootstrap.cc:492] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.547076 12295 tablet_bootstrap.cc:492] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.547426 12295 ts_tablet_manager.cc:1367] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20240502 21:29:31.547514 12294 ts_tablet_manager.cc:1367] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 21:29:31.548270 12295 raft_consensus.cc:348] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.549226 12295 raft_consensus.cc:374] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.549368 12297 tablet_bootstrap.cc:492] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.549104 12294 raft_consensus.cc:348] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.549540 12297 ts_tablet_manager.cc:1367] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20240502 21:29:31.549736 12294 raft_consensus.cc:374] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.549875 12294 raft_consensus.cc:729] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.550209 12294 consensus_queue.cc:260] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.550304 12297 raft_consensus.cc:348] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.550578 12297 raft_consensus.cc:374] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.550714 12297 raft_consensus.cc:729] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.551057 12294 ts_tablet_manager.cc:1398] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.550948 12297 consensus_queue.cc:260] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.551537 12297 ts_tablet_manager.cc:1398] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 21:29:31.551585 12294 tablet_bootstrap.cc:492] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.551837 12297 tablet_bootstrap.cc:492] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.552203 12295 raft_consensus.cc:729] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.552464 12295 consensus_queue.cc:260] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.553136 12295 ts_tablet_manager.cc:1398] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:29:31.553289 12297 tablet_bootstrap.cc:654] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.553458 12294 tablet_bootstrap.cc:654] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.553464 12295 tablet_bootstrap.cc:492] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.556362 12295 tablet_bootstrap.cc:654] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.557751 12297 tablet_bootstrap.cc:492] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.557838 12294 tablet_bootstrap.cc:492] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.557973 12297 ts_tablet_manager.cc:1367] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20240502 21:29:31.558048 12294 ts_tablet_manager.cc:1367] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 21:29:31.558831 12294 raft_consensus.cc:348] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.559092 12294 raft_consensus.cc:374] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.558979 12297 raft_consensus.cc:348] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.559255 12294 raft_consensus.cc:729] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.559267 12297 raft_consensus.cc:374] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.559404 12297 raft_consensus.cc:729] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.559657 12294 consensus_queue.cc:260] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.559715 12297 consensus_queue.cc:260] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.560442 12297 ts_tablet_manager.cc:1398] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:31.560855 12297 tablet_bootstrap.cc:492] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.561242 12294 ts_tablet_manager.cc:1398] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.561549 12294 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.563087 12297 tablet_bootstrap.cc:654] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.563724 12363 consensus_queue.cc:1035] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.570163 12295 tablet_bootstrap.cc:492] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.570163 12297 tablet_bootstrap.cc:492] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.570446 12295 ts_tablet_manager.cc:1367] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20240502 21:29:31.570493 12297 ts_tablet_manager.cc:1367] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20240502 21:29:31.570616 12294 tablet_bootstrap.cc:654] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.572063 12295 raft_consensus.cc:348] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.572571 12297 raft_consensus.cc:348] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.572904 12297 raft_consensus.cc:374] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.572592 12295 raft_consensus.cc:374] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.573087 12297 raft_consensus.cc:729] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.573452 12297 consensus_queue.cc:260] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.573880 12294 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.574080 12294 ts_tablet_manager.cc:1367] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240502 21:29:31.574303 12297 ts_tablet_manager.cc:1398] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 21:29:31.574685 12297 tablet_bootstrap.cc:492] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.574991 12294 raft_consensus.cc:348] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.575242 12294 raft_consensus.cc:374] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.575353 12294 raft_consensus.cc:729] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.573175 12295 raft_consensus.cc:729] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.575742 12295 consensus_queue.cc:260] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.576414 12295 ts_tablet_manager.cc:1398] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20240502 21:29:31.576687 12297 tablet_bootstrap.cc:654] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.576678 12294 consensus_queue.cc:260] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.577224 12294 ts_tablet_manager.cc:1398] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.577608 12295 tablet_bootstrap.cc:492] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.579140 12295 tablet_bootstrap.cc:654] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.580294 12294 tablet_bootstrap.cc:492] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.580657 12448 consensus_queue.cc:1035] T 3f6b3c759a8b4236b3712d5161bdb8dc P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.581771 12294 tablet_bootstrap.cc:654] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.583863 12295 tablet_bootstrap.cc:492] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.584128 12295 ts_tablet_manager.cc:1367] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20240502 21:29:31.585238 12295 raft_consensus.cc:348] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.585558 12295 raft_consensus.cc:374] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.585937 12295 raft_consensus.cc:729] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.586323 12295 consensus_queue.cc:260] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.586547 12297 tablet_bootstrap.cc:492] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.586881 12297 ts_tablet_manager.cc:1367] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20240502 21:29:31.587720 12297 raft_consensus.cc:348] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.587853 12448 raft_consensus.cc:482] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.588163 12297 raft_consensus.cc:374] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.588193 12448 raft_consensus.cc:504] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.588346 12297 raft_consensus.cc:729] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.588757 12297 consensus_queue.cc:260] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.589596 12297 ts_tablet_manager.cc:1398] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.589812 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9bd1fc947f428197d8747df293dffd" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.590046 12297 tablet_bootstrap.cc:492] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.591560 12295 ts_tablet_manager.cc:1398] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20240502 21:29:31.591706 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9bd1fc947f428197d8747df293dffd" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.591806 12297 tablet_bootstrap.cc:654] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.591992 12097 raft_consensus.cc:2379] T 1a9bd1fc947f428197d8747df293dffd P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in this term.
I20240502 21:29:31.593878 12140 leader_election.cc:302] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.594219 12301 consensus_queue.cc:1035] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.594713 12301 raft_consensus.cc:3037] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.588908 12448 leader_election.cc:288] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.594903 12335 raft_consensus.cc:482] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.592010 12295 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.595129 12335 raft_consensus.cc:504] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.596064 12335 leader_election.cc:288] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.596958 12295 tablet_bootstrap.cc:654] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.596972 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db39150ee1094ace88cbc8a2c105985a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.597520 12098 raft_consensus.cc:2379] T db39150ee1094ace88cbc8a2c105985a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 17f04379252d4ecfab3a4916c61a984f in this term.
I20240502 21:29:31.598955 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db39150ee1094ace88cbc8a2c105985a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.601982 12301 raft_consensus.cc:2726] T 1a9bd1fc947f428197d8747df293dffd P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.603519 12335 raft_consensus.cc:482] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.603727 12335 raft_consensus.cc:504] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.604727 12335 leader_election.cc:288] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.605674 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.606055 12098 raft_consensus.cc:2379] T 2924f688055f461ba9c7c0e70197376a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 17f04379252d4ecfab3a4916c61a984f in this term.
I20240502 21:29:31.607158 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2924f688055f461ba9c7c0e70197376a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.607717 12219 leader_election.cc:302] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.608410 12294 tablet_bootstrap.cc:492] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.608664 12294 ts_tablet_manager.cc:1367] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.008s
I20240502 21:29:31.608841 12335 raft_consensus.cc:3037] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.609952 12294 raft_consensus.cc:348] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.610271 12294 raft_consensus.cc:374] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.610440 12294 raft_consensus.cc:729] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.610818 12294 consensus_queue.cc:260] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.608593 12219 leader_election.cc:302] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.611429 12308 raft_consensus.cc:3037] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.612416 12294 ts_tablet_manager.cc:1398] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.612915 12294 tablet_bootstrap.cc:492] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.614914 12308 raft_consensus.cc:2726] T 2924f688055f461ba9c7c0e70197376a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 21:29:31.615175 12066 consensus_peers.cc:485] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Couldn't send request to peer 17f04379252d4ecfab3a4916c61a984f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:31.615551 12294 tablet_bootstrap.cc:654] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.616402 12316 raft_consensus.cc:482] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.619444 12316 raft_consensus.cc:504] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.621740 12316 leader_election.cc:288] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.621796 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e0f94e6b94037b90b833582f3670f" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.622231 12335 raft_consensus.cc:2726] T db39150ee1094ace88cbc8a2c105985a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.622742 12249 raft_consensus.cc:2454] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:31.622771 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e0f94e6b94037b90b833582f3670f" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.623224 12175 raft_consensus.cc:2454] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:31.622759 12251 raft_consensus.cc:3037] T fe982f82c003411e9f9dda42378399f8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.623900 12065 leader_election.cc:302] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.629257 12316 raft_consensus.cc:2781] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.629724 12316 raft_consensus.cc:482] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.629886 12316 raft_consensus.cc:3037] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.630100 12301 consensus_queue.cc:1035] T fe982f82c003411e9f9dda42378399f8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.633945 12366 raft_consensus.cc:482] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.634155 12366 raft_consensus.cc:504] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.634868 12366 leader_election.cc:288] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.635339 12297 tablet_bootstrap.cc:492] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.635591 12297 ts_tablet_manager.cc:1367] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.046s user 0.009s sys 0.017s
I20240502 21:29:31.635624 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e15a0d7054443ea55550d20ee0518e" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.636160 12065 leader_election.cc:341] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.636626 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e15a0d7054443ea55550d20ee0518e" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.637059 12295 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
W20240502 21:29:31.637153 12066 leader_election.cc:341] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.637257 12295 ts_tablet_manager.cc:1367] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.045s user 0.016s sys 0.024s
I20240502 21:29:31.637331 12066 leader_election.cc:302] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.637643 12367 raft_consensus.cc:2726] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.637850 12294 tablet_bootstrap.cc:492] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.638058 12294 ts_tablet_manager.cc:1367] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.010s
I20240502 21:29:31.638316 12295 raft_consensus.cc:348] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.638777 12295 raft_consensus.cc:374] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.638913 12295 raft_consensus.cc:729] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.638856 12294 raft_consensus.cc:348] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.639101 12294 raft_consensus.cc:374] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.639259 12294 raft_consensus.cc:729] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.639160 12295 consensus_queue.cc:260] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.639503 12294 consensus_queue.cc:260] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.640416 12297 raft_consensus.cc:348] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.640689 12297 raft_consensus.cc:374] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.640852 12297 raft_consensus.cc:729] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.641137 12297 consensus_queue.cc:260] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.641589 12294 ts_tablet_manager.cc:1398] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.641834 12297 ts_tablet_manager.cc:1398] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 21:29:31.641957 12294 tablet_bootstrap.cc:492] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.641772 12308 consensus_queue.cc:1035] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.642306 12297 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.642560 12295 ts_tablet_manager.cc:1398] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240502 21:29:31.643870 12294 tablet_bootstrap.cc:654] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.644223 12297 tablet_bootstrap.cc:654] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.646685 12366 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240502 21:29:31.650957 12316 raft_consensus.cc:504] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.651897 12316 leader_election.cc:288] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.652350 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e0f94e6b94037b90b833582f3670f" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:31.652451 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e0f94e6b94037b90b833582f3670f" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.652604 12175 raft_consensus.cc:3037] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.652637 12253 raft_consensus.cc:3037] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.657456 12295 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.659850 12253 raft_consensus.cc:2454] T 494e0f94e6b94037b90b833582f3670f P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:31.660125 12295 tablet_bootstrap.cc:654] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.660575 12065 leader_election.cc:302] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.660969 12316 raft_consensus.cc:2781] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.661168 12316 raft_consensus.cc:686] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:31.660943 12175 raft_consensus.cc:2454] T 494e0f94e6b94037b90b833582f3670f P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:31.662406 12316 consensus_queue.cc:237] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.665815 12297 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.666189 12297 ts_tablet_manager.cc:1367] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.011s
I20240502 21:29:31.675024 12301 raft_consensus.cc:482] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.675748 12297 raft_consensus.cc:348] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.675926 12301 raft_consensus.cc:504] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.676278 12297 raft_consensus.cc:374] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.676512 12297 raft_consensus.cc:729] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.677739 12294 tablet_bootstrap.cc:492] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.677817 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.677950 12294 ts_tablet_manager.cc:1367] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.005s
I20240502 21:29:31.678164 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.678660 12301 leader_election.cc:288] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:31.678800 12142 leader_election.cc:341] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.679072 12096 raft_consensus.cc:2454] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.679026 12297 consensus_queue.cc:260] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.680068 12140 leader_election.cc:302] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.681111 12301 raft_consensus.cc:2781] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.681113 12294 raft_consensus.cc:348] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.681699 12301 raft_consensus.cc:482] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.681758 12297 ts_tablet_manager.cc:1398] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20240502 21:29:31.681864 12301 raft_consensus.cc:3037] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.682364 12297 tablet_bootstrap.cc:492] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.684481 12297 tablet_bootstrap.cc:654] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.685324 12294 raft_consensus.cc:374] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.687603 12295 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.688858 12175 raft_consensus.cc:3037] T a4afa7764f554f62abb539fe13838764 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.691315 12295 ts_tablet_manager.cc:1367] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.018s
I20240502 21:29:31.693576 12301 raft_consensus.cc:504] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.694273 12301 leader_election.cc:288] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.695305 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.695443 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.695537 12097 raft_consensus.cc:3037] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:31.695914 12142 leader_election.cc:341] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.697845 12097 raft_consensus.cc:2454] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.698323 12140 leader_election.cc:302] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.698633 12301 raft_consensus.cc:2781] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.698809 12301 raft_consensus.cc:686] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.699110 12301 consensus_queue.cc:237] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.699793 12294 raft_consensus.cc:729] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.700209 12294 consensus_queue.cc:260] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.701014 12294 ts_tablet_manager.cc:1398] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.023s user 0.011s sys 0.003s
I20240502 21:29:31.701238 12449 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.701442 12294 tablet_bootstrap.cc:492] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.701524 12316 raft_consensus.cc:482] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.701649 12316 raft_consensus.cc:504] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.702242 12316 leader_election.cc:288] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.702805 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.703102 12294 tablet_bootstrap.cc:654] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.703052 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.703456 12065 leader_election.cc:341] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:31.703837 12066 leader_election.cc:341] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.704034 12066 leader_election.cc:302] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.704424 12316 raft_consensus.cc:2726] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.705080 11993 catalog_manager.cc:5484] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.707263 12452 consensus_queue.cc:1035] T a4afa7764f554f62abb539fe13838764 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.706630 12295 raft_consensus.cc:348] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.710187 12295 raft_consensus.cc:374] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.710389 12295 raft_consensus.cc:729] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.710839 12295 consensus_queue.cc:260] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.711757 12295 ts_tablet_manager.cc:1398] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20240502 21:29:31.712270 12295 tablet_bootstrap.cc:492] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.713001 12297 tablet_bootstrap.cc:492] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.713217 12297 ts_tablet_manager.cc:1367] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.002s
I20240502 21:29:31.714313 12297 raft_consensus.cc:348] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.714597 12297 raft_consensus.cc:374] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.714689 12297 raft_consensus.cc:729] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.715019 12297 consensus_queue.cc:260] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.716018 12297 ts_tablet_manager.cc:1398] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20240502 21:29:31.716430 12297 tablet_bootstrap.cc:492] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.716729 12295 tablet_bootstrap.cc:654] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.718703 12297 tablet_bootstrap.cc:654] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.725268 12297 tablet_bootstrap.cc:492] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.725450 12297 ts_tablet_manager.cc:1367] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20240502 21:29:31.726920 12297 raft_consensus.cc:348] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.727221 12297 raft_consensus.cc:374] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.727278 12297 raft_consensus.cc:729] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.727571 12297 consensus_queue.cc:260] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.728329 12297 ts_tablet_manager.cc:1398] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:31.729807 12294 tablet_bootstrap.cc:492] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.729981 12294 ts_tablet_manager.cc:1367] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.013s
I20240502 21:29:31.730443 12295 tablet_bootstrap.cc:492] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.731027 12295 ts_tablet_manager.cc:1367] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.002s
I20240502 21:29:31.731029 12294 raft_consensus.cc:348] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.731424 12294 raft_consensus.cc:374] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.731509 12294 raft_consensus.cc:729] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.731765 12294 consensus_queue.cc:260] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.732046 12297 tablet_bootstrap.cc:492] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.732461 12294 ts_tablet_manager.cc:1398] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:31.732810 12294 tablet_bootstrap.cc:492] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.733083 12295 raft_consensus.cc:348] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.733639 12295 raft_consensus.cc:374] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.733865 12295 raft_consensus.cc:729] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.734259 12295 consensus_queue.cc:260] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.733963 12297 tablet_bootstrap.cc:654] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.735836 12295 ts_tablet_manager.cc:1398] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 21:29:31.736142 12295 tablet_bootstrap.cc:492] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.737329 12295 tablet_bootstrap.cc:654] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.737356 12294 tablet_bootstrap.cc:654] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.737185 11995 catalog_manager.cc:5484] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.740068 12335 raft_consensus.cc:482] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.740252 12335 raft_consensus.cc:504] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.741150 12335 leader_election.cc:288] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.742416 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.742444 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.743137 12096 raft_consensus.cc:2454] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.743772 12216 leader_election.cc:302] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.744308 12335 raft_consensus.cc:2781] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.744415 12335 raft_consensus.cc:482] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.744513 12335 raft_consensus.cc:3037] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.745582 12294 tablet_bootstrap.cc:492] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.745724 12294 ts_tablet_manager.cc:1367] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.003s
I20240502 21:29:31.747126 12294 raft_consensus.cc:348] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.747341 12294 raft_consensus.cc:374] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.747411 12294 raft_consensus.cc:729] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.747608 12294 consensus_queue.cc:260] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.748417 12294 ts_tablet_manager.cc:1398] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.748724 12294 tablet_bootstrap.cc:492] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.749158 12295 tablet_bootstrap.cc:492] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.749279 12295 ts_tablet_manager.cc:1367] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.005s
I20240502 21:29:31.750381 12297 tablet_bootstrap.cc:492] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.750524 12297 ts_tablet_manager.cc:1367] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
W20240502 21:29:31.750489 12219 leader_election.cc:341] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.750859 12295 raft_consensus.cc:348] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.751073 12295 raft_consensus.cc:374] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.751130 12295 raft_consensus.cc:729] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.751170 12297 raft_consensus.cc:348] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.751339 12297 raft_consensus.cc:374] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.751443 12297 raft_consensus.cc:729] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.751328 12295 consensus_queue.cc:260] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.751681 12297 consensus_queue.cc:260] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.751880 12295 ts_tablet_manager.cc:1398] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.752158 12295 tablet_bootstrap.cc:492] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.753233 12361 raft_consensus.cc:482] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.753296 12448 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.753365 12361 raft_consensus.cc:504] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.753472 12297 ts_tablet_manager.cc:1398] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 21:29:31.753684 12448 consensus_queue.cc:1035] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.753799 12295 tablet_bootstrap.cc:654] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.750494 12294 tablet_bootstrap.cc:654] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.753926 12361 leader_election.cc:288] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.756021 12454 consensus_queue.cc:1035] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.756685 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa48c97f4426444585651368f28b265a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.756919 12251 raft_consensus.cc:2454] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.757302 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa48c97f4426444585651368f28b265a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.757500 12097 raft_consensus.cc:2454] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.757876 12140 leader_election.cc:302] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.758322 12363 raft_consensus.cc:2781] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.758419 12363 raft_consensus.cc:482] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.758495 12363 raft_consensus.cc:3037] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.753765 12297 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.761494 12295 tablet_bootstrap.cc:492] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.761682 12295 ts_tablet_manager.cc:1367] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20240502 21:29:31.761926 12363 raft_consensus.cc:504] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.762684 12295 raft_consensus.cc:348] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.762972 12295 raft_consensus.cc:374] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.763056 12295 raft_consensus.cc:729] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.763330 12295 consensus_queue.cc:260] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.762997 12297 tablet_bootstrap.cc:654] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.764106 12295 ts_tablet_manager.cc:1398] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:31.766220 12301 consensus_queue.cc:1035] T db39150ee1094ace88cbc8a2c105985a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.766444 12295 tablet_bootstrap.cc:492] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.764442 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa48c97f4426444585651368f28b265a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.767076 12097 raft_consensus.cc:3037] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.769279 12295 tablet_bootstrap.cc:654] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.769755 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa48c97f4426444585651368f28b265a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.770066 12253 raft_consensus.cc:3037] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.770283 12363 leader_election.cc:288] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.772840 12253 raft_consensus.cc:2454] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.772840 12097 raft_consensus.cc:2454] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.775585 12142 leader_election.cc:302] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.776151 12294 tablet_bootstrap.cc:492] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.776298 12294 ts_tablet_manager.cc:1367] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.012s
I20240502 21:29:31.774433 12335 raft_consensus.cc:504] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.777148 12294 raft_consensus.cc:348] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.777352 12294 raft_consensus.cc:374] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.777420 12294 raft_consensus.cc:729] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.777638 12294 consensus_queue.cc:260] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.777848 12363 raft_consensus.cc:2781] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.777959 12363 raft_consensus.cc:686] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.779835 12301 consensus_queue.cc:1035] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.780002 12297 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.780161 12297 ts_tablet_manager.cc:1367] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.003s
I20240502 21:29:31.780901 12297 raft_consensus.cc:348] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.781100 12297 raft_consensus.cc:374] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.781158 12297 raft_consensus.cc:729] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.781327 12294 ts_tablet_manager.cc:1398] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:29:31.781683 12294 tablet_bootstrap.cc:492] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.781641 12301 raft_consensus.cc:482] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.781865 12301 raft_consensus.cc:504] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.782567 12301 leader_election.cc:288] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.783138 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aef08a96a4e49f6bc1d3fc3175d1685" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.783349 12251 raft_consensus.cc:2454] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.783548 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aef08a96a4e49f6bc1d3fc3175d1685" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.783737 12142 leader_election.cc:302] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:31.783782 12097 raft_consensus.cc:2454] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.784044 12301 raft_consensus.cc:2781] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.784134 12301 raft_consensus.cc:482] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.784171 12294 tablet_bootstrap.cc:654] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.784214 12301 raft_consensus.cc:3037] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.781369 12297 consensus_queue.cc:260] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.786304 12363 consensus_queue.cc:237] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.786439 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.786796 12097 raft_consensus.cc:3037] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.788219 12335 leader_election.cc:288] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.788424 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
W20240502 21:29:31.788973 12219 leader_election.cc:341] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.791733 12361 consensus_queue.cc:1035] T 2924f688055f461ba9c7c0e70197376a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.792878 12294 tablet_bootstrap.cc:492] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.793151 12294 ts_tablet_manager.cc:1367] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.001s
I20240502 21:29:31.793561 12297 ts_tablet_manager.cc:1398] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20240502 21:29:31.793603 12316 raft_consensus.cc:482] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.793710 12316 raft_consensus.cc:504] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.793857 12297 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.794876 12316 leader_election.cc:288] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.795436 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9adc5b5ce04374bf15c9088b4397d8" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.795941 12065 leader_election.cc:341] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.796115 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9adc5b5ce04374bf15c9088b4397d8" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.796993 12066 leader_election.cc:341] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.797183 12066 leader_election.cc:302] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.797719 12316 raft_consensus.cc:2726] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.802347 12301 raft_consensus.cc:504] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.802958 12294 raft_consensus.cc:348] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.803222 12294 raft_consensus.cc:374] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.803315 12294 raft_consensus.cc:729] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.803606 12294 consensus_queue.cc:260] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.804370 12294 ts_tablet_manager.cc:1398] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20240502 21:29:31.804765 12294 tablet_bootstrap.cc:492] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.804442 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aef08a96a4e49f6bc1d3fc3175d1685" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.806546 12099 raft_consensus.cc:3037] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.803218 12301 leader_election.cc:288] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.809765 12251 raft_consensus.cc:3037] T 9e5d3606b31841f6986c1c0954eb9332 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.810881 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aef08a96a4e49f6bc1d3fc3175d1685" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.813930 12253 raft_consensus.cc:3037] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.815234 12335 consensus_queue.cc:1035] T b0766fbf50cb4b17a55ee3a21de8248c P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.816915 12297 tablet_bootstrap.cc:654] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.822750 12099 raft_consensus.cc:2454] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.824201 12140 leader_election.cc:302] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.824579 12301 raft_consensus.cc:2781] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.824755 12301 raft_consensus.cc:686] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:31.825201 12301 consensus_queue.cc:237] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.827314 12295 tablet_bootstrap.cc:492] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.828418 12361 consensus_queue.cc:1035] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.823807 12097 raft_consensus.cc:2454] T 81abff8d58dc4c69b08ac2a161748b5e P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:31.832005 12335 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.832345 12253 raft_consensus.cc:2454] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.832453 12216 leader_election.cc:302] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.832775 12335 raft_consensus.cc:2781] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.832976 12335 raft_consensus.cc:686] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:31.833282 12335 consensus_queue.cc:237] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.834183 12295 ts_tablet_manager.cc:1367] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.068s user 0.011s sys 0.021s
I20240502 21:29:31.837862 11993 catalog_manager.cc:5484] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.838338 12363 raft_consensus.cc:482] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.838481 12363 raft_consensus.cc:504] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.839535 12363 leader_election.cc:288] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.839588 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c958a20984244369aeed6a8442d51dd" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.839833 12253 raft_consensus.cc:2454] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.840258 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c958a20984244369aeed6a8442d51dd" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.840457 12097 raft_consensus.cc:2454] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:31.840811 12140 leader_election.cc:302] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.841077 12363 raft_consensus.cc:2781] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.841166 12363 raft_consensus.cc:482] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.841269 12363 raft_consensus.cc:3037] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.844190 12363 raft_consensus.cc:504] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.845503 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c958a20984244369aeed6a8442d51dd" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.845746 12096 raft_consensus.cc:3037] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.845983 12363 leader_election.cc:288] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.846496 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c958a20984244369aeed6a8442d51dd" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:31.846729 12253 raft_consensus.cc:3037] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.851317 12295 raft_consensus.cc:348] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.851629 12295 raft_consensus.cc:374] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.851698 12295 raft_consensus.cc:729] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.852051 12295 consensus_queue.cc:260] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.852859 12295 ts_tablet_manager.cc:1398] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20240502 21:29:31.852905 12294 tablet_bootstrap.cc:654] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.853273 12295 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.856223 12335 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.862468 12316 raft_consensus.cc:482] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.862706 12316 raft_consensus.cc:504] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.863581 12316 leader_election.cc:288] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.864357 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d4e5a4db934406b9f995442ec6bcd1" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.865058 12065 leader_election.cc:341] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.865705 12316 raft_consensus.cc:482] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.865648 12449 raft_consensus.cc:482] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.866128 12449 raft_consensus.cc:504] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.866925 12449 leader_election.cc:288] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.865847 12316 raft_consensus.cc:504] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.867772 12097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a147afb591461387bfe6ed7587a6ba" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.867864 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a147afb591461387bfe6ed7587a6ba" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.869052 12366 raft_consensus.cc:482] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.869427 12366 raft_consensus.cc:504] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.869609 12316 leader_election.cc:288] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.869939 12366 leader_election.cc:288] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.869058 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d4e5a4db934406b9f995442ec6bcd1" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.870682 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528a557fdd90485d91077d2a61095878" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.871621 12065 leader_election.cc:341] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.872290 12140 leader_election.cc:302] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.872650 12363 raft_consensus.cc:2726] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.873131 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8b568beaec444faa5e779e0b2059b1" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.873759 12066 leader_election.cc:341] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.874747 12096 raft_consensus.cc:2454] T 0c958a20984244369aeed6a8442d51dd P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:31.875504 12140 leader_election.cc:302] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:31.870934 12249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8b568beaec444faa5e779e0b2059b1" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.876152 12065 leader_election.cc:341] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.876334 12361 raft_consensus.cc:2781] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:31.876504 12361 raft_consensus.cc:686] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
W20240502 21:29:31.876709 12066 leader_election.cc:341] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.876866 12066 leader_election.cc:302] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.877166 12366 raft_consensus.cc:2726] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.878158 12065 leader_election.cc:302] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.878546 12366 raft_consensus.cc:2726] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.876838 12361 consensus_queue.cc:237] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.874133 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528a557fdd90485d91077d2a61095878" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.894645 12449 consensus_queue.cc:1035] T 9e5d3606b31841f6986c1c0954eb9332 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.897553 12253 raft_consensus.cc:2454] T 0c958a20984244369aeed6a8442d51dd P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
W20240502 21:29:31.900251 12066 leader_election.cc:341] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.900413 12066 leader_election.cc:302] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.901088 12316 raft_consensus.cc:2726] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.902742 12295 tablet_bootstrap.cc:654] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.903601 11993 catalog_manager.cc:5484] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:31.907495 12334 consensus_queue.cc:1035] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240502 21:29:31.908236 12335 consensus_queue.cc:1035] T d6c0955962874018b574931aea7bd0ce P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:31.913205 12294 tablet_bootstrap.cc:492] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.913406 12294 ts_tablet_manager.cc:1367] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.109s user 0.037s sys 0.004s
I20240502 21:29:31.914580 12297 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.914752 12297 ts_tablet_manager.cc:1367] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.121s user 0.005s sys 0.012s
I20240502 21:29:31.916124 12294 raft_consensus.cc:348] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.916411 12294 raft_consensus.cc:374] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.916465 12294 raft_consensus.cc:729] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.918022 12294 consensus_queue.cc:260] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.918192 11995 catalog_manager.cc:5484] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.918640 12294 ts_tablet_manager.cc:1398] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:29:31.918923 12294 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.918936 12295 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.919095 12295 ts_tablet_manager.cc:1367] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.066s user 0.009s sys 0.001s
I20240502 21:29:31.919867 12295 raft_consensus.cc:348] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.920092 12295 raft_consensus.cc:374] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.920166 12295 raft_consensus.cc:729] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.920403 12295 consensus_queue.cc:260] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.921538 12295 ts_tablet_manager.cc:1398] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:31.921854 12295 tablet_bootstrap.cc:492] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.923300 12297 raft_consensus.cc:348] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.923614 12297 raft_consensus.cc:374] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.923753 12297 raft_consensus.cc:729] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.924096 12297 consensus_queue.cc:260] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.924731 12297 ts_tablet_manager.cc:1398] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.010s user 0.000s sys 0.003s
I20240502 21:29:31.925074 12297 tablet_bootstrap.cc:492] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.927019 12294 tablet_bootstrap.cc:654] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.934903 12316 raft_consensus.cc:482] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.935123 12316 raft_consensus.cc:504] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.936340 12316 leader_election.cc:288] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.936880 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.936894 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.937392 12066 leader_election.cc:341] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:31.937659 12065 leader_election.cc:341] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.937757 12065 leader_election.cc:302] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.938267 12316 raft_consensus.cc:2726] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.940193 12294 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.940361 12294 ts_tablet_manager.cc:1367] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.000s
I20240502 21:29:31.940537 11993 catalog_manager.cc:5484] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:31.941190 12294 raft_consensus.cc:348] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.941421 12294 raft_consensus.cc:374] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.941509 12294 raft_consensus.cc:729] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.941615 12295 tablet_bootstrap.cc:654] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.941766 12294 consensus_queue.cc:260] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.942515 12294 ts_tablet_manager.cc:1398] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:31.942963 12294 tablet_bootstrap.cc:492] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.943324 12297 tablet_bootstrap.cc:654] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.944819 12294 tablet_bootstrap.cc:654] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.952451 12297 tablet_bootstrap.cc:492] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.952646 12297 ts_tablet_manager.cc:1367] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.007s
I20240502 21:29:31.954440 12297 raft_consensus.cc:348] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.954676 12297 raft_consensus.cc:374] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.954741 12297 raft_consensus.cc:729] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.954991 12297 consensus_queue.cc:260] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.955987 12295 tablet_bootstrap.cc:492] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.956259 12295 ts_tablet_manager.cc:1367] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.005s
I20240502 21:29:31.956668 12297 ts_tablet_manager.cc:1398] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:31.956969 12316 raft_consensus.cc:482] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.957115 12316 raft_consensus.cc:504] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.957384 12295 raft_consensus.cc:348] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.957564 12295 raft_consensus.cc:374] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.957636 12295 raft_consensus.cc:729] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.957901 12316 leader_election.cc:288] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.957008 12297 tablet_bootstrap.cc:492] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:31.958271 12295 consensus_queue.cc:260] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.958813 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e5a3ff13f40629080ec7036eb39ef" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.959641 12066 leader_election.cc:341] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.959877 12297 tablet_bootstrap.cc:654] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.962160 12294 tablet_bootstrap.cc:492] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:31.959013 12295 ts_tablet_manager.cc:1398] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:31.962363 12294 ts_tablet_manager.cc:1367] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.011s
I20240502 21:29:31.962200 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e5a3ff13f40629080ec7036eb39ef" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.962767 12295 tablet_bootstrap.cc:492] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.963310 12294 raft_consensus.cc:348] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.963567 12294 raft_consensus.cc:374] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.963657 12294 raft_consensus.cc:729] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.963928 12294 consensus_queue.cc:260] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.964640 12294 ts_tablet_manager.cc:1398] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:31.965339 12295 tablet_bootstrap.cc:654] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.965315 12335 raft_consensus.cc:482] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.965663 12335 raft_consensus.cc:504] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.965961 12366 raft_consensus.cc:482] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.966197 12366 raft_consensus.cc:504] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
W20240502 21:29:31.963376 12065 leader_election.cc:341] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.967299 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.967803 12219 leader_election.cc:341] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.968441 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.968731 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b211abd4317f4be485f9db73fd55629e" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.969293 12335 leader_election.cc:288] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.969539 12294 tablet_bootstrap.cc:492] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:31.970250 12099 raft_consensus.cc:2454] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:31.970698 12065 leader_election.cc:302] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.970707 12216 leader_election.cc:302] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
W20240502 21:29:31.971405 12066 leader_election.cc:341] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.971554 12316 raft_consensus.cc:2726] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.971627 12367 raft_consensus.cc:482] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.971930 12316 raft_consensus.cc:482] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:31.972096 12316 raft_consensus.cc:504] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.972223 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b211abd4317f4be485f9db73fd55629e" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.971810 12367 raft_consensus.cc:504] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.972801 12316 leader_election.cc:288] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.972991 12367 leader_election.cc:288] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
W20240502 21:29:31.973363 12065 leader_election.cc:341] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.973529 12065 leader_election.cc:302] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.973695 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6c003ea2884ca9a1827814a4930742" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:31.973850 12367 raft_consensus.cc:2726] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.973661 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a7f75fa74945c09343353c01f4b31b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:31.974287 12065 leader_election.cc:341] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.973006 12294 tablet_bootstrap.cc:654] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:31.975040 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6c003ea2884ca9a1827814a4930742" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.975435 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a7f75fa74945c09343353c01f4b31b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:31.975988 12066 leader_election.cc:341] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.976154 12066 leader_election.cc:302] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
W20240502 21:29:31.976756 12066 leader_election.cc:341] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.977001 12367 raft_consensus.cc:2726] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.977535 12334 raft_consensus.cc:2781] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:31.977697 12334 raft_consensus.cc:482] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:31.977843 12334 raft_consensus.cc:3037] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.980017 12366 leader_election.cc:288] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:31.982632 12297 tablet_bootstrap.cc:492] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:31.982868 12297 ts_tablet_manager.cc:1367] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.000s
I20240502 21:29:31.983111 12335 raft_consensus.cc:482] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240502 21:29:31.983736 12065 leader_election.cc:341] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:31.983927 12065 leader_election.cc:302] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:31.985054 12335 raft_consensus.cc:504] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.986063 12335 leader_election.cc:288] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.988145 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:31.988577 12366 raft_consensus.cc:2726] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.989033 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad85ac6e9a824608ad426e1453c70237" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:31.990180 12099 raft_consensus.cc:2379] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in current term 1: Already voted for candidate 17f04379252d4ecfab3a4916c61a984f in this term.
I20240502 21:29:31.991991 12216 leader_election.cc:302] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:31.992400 12295 tablet_bootstrap.cc:492] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:31.992691 12335 raft_consensus.cc:3037] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.992866 12297 raft_consensus.cc:348] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.993095 12297 raft_consensus.cc:374] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.993260 12297 raft_consensus.cc:729] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.993436 12334 raft_consensus.cc:504] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.995649 12335 raft_consensus.cc:2726] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:31.995879 12295 ts_tablet_manager.cc:1367] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.019s
I20240502 21:29:31.996124 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:31.996479 12099 raft_consensus.cc:3037] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:31.996814 12295 raft_consensus.cc:348] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.997069 12295 raft_consensus.cc:374] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:31.997191 12295 raft_consensus.cc:729] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:31.997354 12334 leader_election.cc:288] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:31.995951 12297 consensus_queue.cc:260] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.997448 12295 consensus_queue.cc:260] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:31.998052 12295 ts_tablet_manager.cc:1398] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:31.998410 12295 tablet_bootstrap.cc:492] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:31.998059 12297 ts_tablet_manager.cc:1398] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.015s user 0.002s sys 0.001s
I20240502 21:29:31.999779 12297 tablet_bootstrap.cc:492] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.000195 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.000329 12295 tablet_bootstrap.cc:654] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
W20240502 21:29:32.000996 12219 leader_election.cc:341] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.004001 12099 raft_consensus.cc:2454] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.004972 12361 raft_consensus.cc:482] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.005167 12361 raft_consensus.cc:504] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.005811 12297 tablet_bootstrap.cc:654] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.006117 12361 leader_election.cc:288] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.007189 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d4e5a4db934406b9f995442ec6bcd1" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:32.007778 12142 leader_election.cc:341] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.008075 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d4e5a4db934406b9f995442ec6bcd1" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.008249 12216 leader_election.cc:302] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.008319 12098 raft_consensus.cc:2454] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.008662 12294 tablet_bootstrap.cc:492] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.008833 12294 ts_tablet_manager.cc:1367] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.040s user 0.023s sys 0.009s
I20240502 21:29:32.008924 12140 leader_election.cc:302] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.009008 12334 raft_consensus.cc:2781] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.009341 12334 raft_consensus.cc:686] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.009410 12361 raft_consensus.cc:2781] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.009550 12361 raft_consensus.cc:482] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.009645 12361 raft_consensus.cc:3037] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.009758 12294 raft_consensus.cc:348] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.010000 12294 raft_consensus.cc:374] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.010108 12294 raft_consensus.cc:729] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.010391 12294 consensus_queue.cc:260] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.012573 12294 ts_tablet_manager.cc:1398] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:32.012913 12294 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.014796 12294 tablet_bootstrap.cc:654] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.014988 12295 tablet_bootstrap.cc:492] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.015168 12295 ts_tablet_manager.cc:1367] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.010s
I20240502 21:29:32.009771 12334 consensus_queue.cc:237] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.016033 12295 raft_consensus.cc:348] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.016260 12295 raft_consensus.cc:374] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.016374 12295 raft_consensus.cc:729] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.016564 12361 raft_consensus.cc:504] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.016624 12295 consensus_queue.cc:260] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.017287 12295 ts_tablet_manager.cc:1398] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:32.017625 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d4e5a4db934406b9f995442ec6bcd1" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.017730 12295 tablet_bootstrap.cc:492] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.017858 12099 raft_consensus.cc:3037] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.019728 12295 tablet_bootstrap.cc:654] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.021063 12099 raft_consensus.cc:2454] T 73d4e5a4db934406b9f995442ec6bcd1 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.021909 12140 leader_election.cc:302] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.022212 12297 tablet_bootstrap.cc:492] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.022320 12449 raft_consensus.cc:2781] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.022440 12297 ts_tablet_manager.cc:1367] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20240502 21:29:32.022500 12449 raft_consensus.cc:686] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.022868 12449 consensus_queue.cc:237] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.025487 12308 raft_consensus.cc:482] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.025588 12297 raft_consensus.cc:348] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.025743 12308 raft_consensus.cc:504] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.025943 12297 raft_consensus.cc:374] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.026051 12297 raft_consensus.cc:729] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.026387 12297 consensus_queue.cc:260] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.026625 12308 leader_election.cc:288] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.027352 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834684d85d644236aab5229b549c428a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.027638 12174 raft_consensus.cc:2454] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.028169 12219 leader_election.cc:302] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.028914 12297 ts_tablet_manager.cc:1398] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20240502 21:29:32.029783 12297 tablet_bootstrap.cc:492] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.029964 12335 raft_consensus.cc:2781] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.030119 12335 raft_consensus.cc:482] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.030261 12335 raft_consensus.cc:3037] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.033056 12297 tablet_bootstrap.cc:654] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.036301 12366 raft_consensus.cc:482] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.036522 12366 raft_consensus.cc:504] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.037443 12366 leader_election.cc:288] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.038201 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbfbbbb346f413782d97580adfa82c1" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.038203 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbfbbbb346f413782d97580adfa82c1" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:32.039197 12066 leader_election.cc:341] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:32.039623 12065 leader_election.cc:341] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.039606 12308 consensus_queue.cc:1035] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.039810 12065 leader_election.cc:302] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.040567 12366 raft_consensus.cc:2726] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.042661 12297 tablet_bootstrap.cc:492] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.042863 12297 ts_tablet_manager.cc:1367] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20240502 21:29:32.043732 12297 raft_consensus.cc:348] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.044068 12297 raft_consensus.cc:374] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.044207 12297 raft_consensus.cc:729] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.044625 12297 consensus_queue.cc:260] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.044875 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834684d85d644236aab5229b549c428a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.045122 12099 raft_consensus.cc:2454] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.045868 12297 ts_tablet_manager.cc:1398] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:32.046329 12297 tablet_bootstrap.cc:492] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.050746 12297 tablet_bootstrap.cc:654] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.052057 12295 tablet_bootstrap.cc:492] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.052246 12295 ts_tablet_manager.cc:1367] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.035s user 0.016s sys 0.016s
I20240502 21:29:32.052520 12361 leader_election.cc:288] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.053180 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d4e5a4db934406b9f995442ec6bcd1" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.053215 12295 raft_consensus.cc:348] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.053397 12295 raft_consensus.cc:374] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.053421 12251 raft_consensus.cc:3037] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.053447 12295 raft_consensus.cc:729] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.053735 12295 consensus_queue.cc:260] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.054510 12295 ts_tablet_manager.cc:1398] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.054883 12295 tablet_bootstrap.cc:492] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.057446 12251 raft_consensus.cc:2454] T 73d4e5a4db934406b9f995442ec6bcd1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.057818 12295 tablet_bootstrap.cc:654] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.059744 12367 raft_consensus.cc:482] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.059929 12367 raft_consensus.cc:504] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.060760 12297 tablet_bootstrap.cc:492] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.060853 12367 leader_election.cc:288] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.061100 12297 ts_tablet_manager.cc:1367] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20240502 21:29:32.062151 12297 raft_consensus.cc:348] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.062512 12297 raft_consensus.cc:374] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.062659 12297 raft_consensus.cc:729] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.063035 12297 consensus_queue.cc:260] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.063360 12361 raft_consensus.cc:482] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.063472 12361 raft_consensus.cc:504] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.063643 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8307fd623a499fa05cd9eca238ddc2" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.064042 12297 ts_tablet_manager.cc:1398] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:32.064321 12361 leader_election.cc:288] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.064384 12297 tablet_bootstrap.cc:492] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
W20240502 21:29:32.064711 12066 leader_election.cc:341] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.065158 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9adc5b5ce04374bf15c9088b4397d8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.065393 12253 raft_consensus.cc:2454] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.065834 12142 leader_election.cc:302] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.066174 12361 raft_consensus.cc:2781] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.066264 12361 raft_consensus.cc:482] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.066349 12361 raft_consensus.cc:3037] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.070076 12297 tablet_bootstrap.cc:654] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.070092 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9adc5b5ce04374bf15c9088b4397d8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.070411 12099 raft_consensus.cc:2454] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.071422 12361 raft_consensus.cc:504] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.072671 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9adc5b5ce04374bf15c9088b4397d8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.073071 12099 raft_consensus.cc:3037] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.076826 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8307fd623a499fa05cd9eca238ddc2" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:32.077504 12065 leader_election.cc:341] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.077664 12065 leader_election.cc:302] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.078189 12367 raft_consensus.cc:2726] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.081039 12363 raft_consensus.cc:482] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.081238 12363 raft_consensus.cc:504] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.082077 12363 leader_election.cc:288] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.083232 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e15a0d7054443ea55550d20ee0518e" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.083499 12096 raft_consensus.cc:2454] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.084048 12140 leader_election.cc:302] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.084391 12363 raft_consensus.cc:2781] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.084497 12363 raft_consensus.cc:482] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.084587 12363 raft_consensus.cc:3037] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.088524 12363 raft_consensus.cc:504] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.089655 12363 leader_election.cc:288] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.090201 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e15a0d7054443ea55550d20ee0518e" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.090430 12096 raft_consensus.cc:3037] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.094213 12096 raft_consensus.cc:2454] T 17e15a0d7054443ea55550d20ee0518e P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.094947 12140 leader_election.cc:302] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.098186 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e15a0d7054443ea55550d20ee0518e" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.098488 12253 raft_consensus.cc:3037] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.102298 12253 raft_consensus.cc:2454] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.105741 12448 raft_consensus.cc:2781] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.105934 12448 raft_consensus.cc:686] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.106283 12448 consensus_queue.cc:237] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.111482 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e15a0d7054443ea55550d20ee0518e" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.112977 12251 raft_consensus.cc:2362] T 17e15a0d7054443ea55550d20ee0518e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1. Re-sending same reply.
I20240502 21:29:32.113885 12367 raft_consensus.cc:482] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.113945 12308 consensus_queue.cc:1035] T 1a9bd1fc947f428197d8747df293dffd P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.114068 12367 raft_consensus.cc:504] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.114914 12367 leader_election.cc:288] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.115764 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb074bc2833742dda1e8e68ca5f2c8e7" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:32.116348 12065 leader_election.cc:341] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.119071 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb074bc2833742dda1e8e68ca5f2c8e7" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:32.127956 12066 leader_election.cc:341] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.128219 12066 leader_election.cc:302] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.128815 12367 raft_consensus.cc:2726] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.142525 12099 raft_consensus.cc:2454] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.143180 12140 leader_election.cc:302] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.143788 12363 raft_consensus.cc:2781] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.144089 12363 raft_consensus.cc:686] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.144531 12363 consensus_queue.cc:237] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.146543 12361 leader_election.cc:288] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.150015 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9adc5b5ce04374bf15c9088b4397d8" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.150363 12251 raft_consensus.cc:3037] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.151856 12361 raft_consensus.cc:482] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.152081 12361 raft_consensus.cc:504] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.153687 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528a557fdd90485d91077d2a61095878" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.154100 12096 raft_consensus.cc:2454] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.155179 12140 leader_election.cc:302] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.155551 12301 raft_consensus.cc:2781] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.155653 12301 raft_consensus.cc:482] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.155745 12301 raft_consensus.cc:3037] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.160761 12301 raft_consensus.cc:504] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.162513 12301 leader_election.cc:288] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.164283 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528a557fdd90485d91077d2a61095878" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.164691 12099 raft_consensus.cc:3037] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.163952 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528a557fdd90485d91077d2a61095878" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.165315 12253 raft_consensus.cc:3037] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.155686 12251 raft_consensus.cc:2454] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.173424 12253 raft_consensus.cc:2454] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.179152 12361 leader_election.cc:288] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.184656 12099 raft_consensus.cc:2454] T 528a557fdd90485d91077d2a61095878 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.214509 12367 consensus_queue.cc:1035] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 21:29:32.230394 12046 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240502 21:29:32.234037 12294 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.234346 12294 ts_tablet_manager.cc:1367] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.222s user 0.033s sys 0.068s
I20240502 21:29:32.234800 11993 catalog_manager.cc:5484] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.235390 12294 raft_consensus.cc:348] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.235606 12294 raft_consensus.cc:374] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.235677 12294 raft_consensus.cc:729] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.235917 12294 consensus_queue.cc:260] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.236302 12295 tablet_bootstrap.cc:492] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.236496 12295 ts_tablet_manager.cc:1367] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.182s user 0.082s sys 0.093s
W20240502 21:29:32.234113 12142 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.185.67:34725, user_credentials={real_user=slave}} blocked reactor thread for 63341.7us
I20240502 21:29:32.237295 12295 raft_consensus.cc:348] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.237573 12295 raft_consensus.cc:374] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.237677 12295 raft_consensus.cc:729] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.238260 12142 leader_election.cc:302] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.238389 12177 raft_consensus.cc:3037] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.240775 12555 raft_consensus.cc:2781] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.241675 12366 consensus_queue.cc:1035] T 494e0f94e6b94037b90b833582f3670f P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.237895 12295 consensus_queue.cc:260] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.242199 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528a557fdd90485d91077d2a61095878" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.243271 12253 raft_consensus.cc:2362] T 528a557fdd90485d91077d2a61095878 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1. Re-sending same reply.
I20240502 21:29:32.243350 12294 ts_tablet_manager.cc:1398] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20240502 21:29:32.243796 12294 tablet_bootstrap.cc:492] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.245232 12555 raft_consensus.cc:686] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.245600 12555 consensus_queue.cc:237] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.246230 12297 tablet_bootstrap.cc:492] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.246416 12297 ts_tablet_manager.cc:1367] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.182s user 0.042s sys 0.050s
I20240502 21:29:32.248075 11996 catalog_manager.cc:5484] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.249028 12294 tablet_bootstrap.cc:654] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.250169 12297 raft_consensus.cc:348] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.250381 12297 raft_consensus.cc:374] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.250442 12297 raft_consensus.cc:729] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.251857 12295 ts_tablet_manager.cc:1398] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20240502 21:29:32.257227 12295 tablet_bootstrap.cc:492] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.259572 12295 tablet_bootstrap.cc:654] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.259932 11996 catalog_manager.cc:5484] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
W20240502 21:29:32.256790 12140 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.185.65:44241, user_credentials={real_user=slave}} blocked reactor thread for 84492.8us
I20240502 21:29:32.250634 12297 consensus_queue.cc:260] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.262553 11996 catalog_manager.cc:5484] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.262912 12297 ts_tablet_manager.cc:1398] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20240502 21:29:32.263410 12297 tablet_bootstrap.cc:492] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.263353 11996 catalog_manager.cc:5484] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.264230 12361 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.267643 12297 tablet_bootstrap.cc:654] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.270080 12361 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.271762 12361 consensus_queue.cc:1035] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.279836 12334 raft_consensus.cc:482] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.280095 12334 raft_consensus.cc:504] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.280990 12334 leader_election.cc:288] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.281616 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.281904 12096 raft_consensus.cc:2454] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.282498 12216 leader_election.cc:302] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.283136 12334 raft_consensus.cc:2781] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.283249 12334 raft_consensus.cc:482] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.283332 12334 raft_consensus.cc:3037] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.283818 12297 tablet_bootstrap.cc:492] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.284066 12297 ts_tablet_manager.cc:1367] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.001s
I20240502 21:29:32.285102 12297 raft_consensus.cc:348] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.285398 12297 raft_consensus.cc:374] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.285470 12297 raft_consensus.cc:729] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.285735 12297 consensus_queue.cc:260] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.287250 12363 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.290416 12297 ts_tablet_manager.cc:1398] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 21:29:32.290839 12297 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.293566 12363 consensus_queue.cc:1035] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.296855 12297 tablet_bootstrap.cc:654] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.300424 12334 raft_consensus.cc:504] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.301954 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.302201 12100 raft_consensus.cc:3037] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.306103 12100 raft_consensus.cc:2454] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.308897 12216 leader_election.cc:302] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.309207 12308 raft_consensus.cc:2781] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.309324 12308 raft_consensus.cc:686] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.309584 12308 consensus_queue.cc:237] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.314335 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
W20240502 21:29:32.315026 12219 leader_election.cc:341] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.315557 12334 leader_election.cc:288] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.322476 12335 raft_consensus.cc:504] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.323235 12448 consensus_queue.cc:1035] T 6aef08a96a4e49f6bc1d3fc3175d1685 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.324026 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834684d85d644236aab5229b549c428a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.324199 12100 raft_consensus.cc:3037] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.325764 12556 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.328034 12556 consensus_queue.cc:1035] T 0c958a20984244369aeed6a8442d51dd P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.328599 12454 consensus_queue.cc:1035] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.332504 12556 raft_consensus.cc:482] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.332661 12556 raft_consensus.cc:504] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.333602 12556 leader_election.cc:288] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.334522 12250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.337832 12556 raft_consensus.cc:482] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.338019 12556 raft_consensus.cc:504] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.338851 12556 leader_election.cc:288] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.339537 12250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.346823 12100 raft_consensus.cc:2454] T 834684d85d644236aab5229b549c428a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.346743 12093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81abff8d58dc4c69b08ac2a161748b5e" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.347466 12216 leader_election.cc:302] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.350198 12452 raft_consensus.cc:2781] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.350411 12452 raft_consensus.cc:686] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.350788 12452 consensus_queue.cc:237] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.353291 12140 leader_election.cc:302] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.353969 12556 raft_consensus.cc:2726] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.355438 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.359050 12100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fbdb33d94f4dd196663efa7f5b1cd7" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.359338 12100 raft_consensus.cc:2379] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in this term.
I20240502 21:29:32.361389 12177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834684d85d644236aab5229b549c428a" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.361788 12177 raft_consensus.cc:3037] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.362545 12140 leader_election.cc:302] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.363013 12363 raft_consensus.cc:3037] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.365254 12177 raft_consensus.cc:2454] T 834684d85d644236aab5229b549c428a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.365607 12363 raft_consensus.cc:2726] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.366739 12335 leader_election.cc:288] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
W20240502 21:29:32.367833 12219 leader_election.cc:341] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.374987 12294 tablet_bootstrap.cc:492] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.375208 12294 ts_tablet_manager.cc:1367] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.131s user 0.047s sys 0.071s
I20240502 21:29:32.376292 12294 raft_consensus.cc:348] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.376587 12294 raft_consensus.cc:374] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.376653 12294 raft_consensus.cc:729] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.376963 12294 consensus_queue.cc:260] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.377874 12294 ts_tablet_manager.cc:1398] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 21:29:32.378273 12294 tablet_bootstrap.cc:492] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.382189 12294 tablet_bootstrap.cc:654] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.386186 12366 raft_consensus.cc:482] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.386386 12366 raft_consensus.cc:504] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.388531 12366 leader_election.cc:288] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.389654 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5655964c244669bd1b644c9399cdec" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:32.390367 12066 leader_election.cc:341] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.392309 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5655964c244669bd1b644c9399cdec" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.393440 12253 raft_consensus.cc:2454] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.394282 12294 tablet_bootstrap.cc:492] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.394454 12294 ts_tablet_manager.cc:1367] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.010s
I20240502 21:29:32.394593 12065 leader_election.cc:302] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.395354 12366 raft_consensus.cc:2781] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.395457 12366 raft_consensus.cc:482] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.395540 12366 raft_consensus.cc:3037] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.395640 12308 raft_consensus.cc:482] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.395818 12308 raft_consensus.cc:504] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.397244 12308 leader_election.cc:288] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.398589 12295 tablet_bootstrap.cc:492] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.398721 12295 ts_tablet_manager.cc:1367] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.142s user 0.033s sys 0.028s
I20240502 21:29:32.398816 12366 raft_consensus.cc:504] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.399497 12295 raft_consensus.cc:348] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.399724 12295 raft_consensus.cc:374] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.399793 12295 raft_consensus.cc:729] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.400035 12295 consensus_queue.cc:260] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.400645 12295 ts_tablet_manager.cc:1398] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.400687 12176 raft_consensus.cc:3037] T 81abff8d58dc4c69b08ac2a161748b5e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.400929 12295 tablet_bootstrap.cc:492] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.401810 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8b568beaec444faa5e779e0b2059b1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.402021 12096 raft_consensus.cc:2454] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.402396 12216 leader_election.cc:302] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.402671 12452 raft_consensus.cc:2781] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.402751 12452 raft_consensus.cc:482] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.402813 12452 raft_consensus.cc:3037] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.404692 11994 catalog_manager.cc:5484] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.405202 11994 catalog_manager.cc:5484] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.406711 12366 leader_election.cc:288] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.406875 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5655964c244669bd1b644c9399cdec" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
W20240502 21:29:32.407755 12066 leader_election.cc:341] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.408149 12295 tablet_bootstrap.cc:654] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.408597 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5655964c244669bd1b644c9399cdec" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.408846 12251 raft_consensus.cc:3037] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.410419 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8b568beaec444faa5e779e0b2059b1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.410673 12174 raft_consensus.cc:2454] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.414110 12557 raft_consensus.cc:482] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.414259 12557 raft_consensus.cc:504] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.417907 12335 consensus_queue.cc:1035] T 81abff8d58dc4c69b08ac2a161748b5e P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.419231 12294 raft_consensus.cc:348] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.419518 12294 raft_consensus.cc:374] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.419595 12294 raft_consensus.cc:729] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.421213 12294 consensus_queue.cc:260] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.428850 12294 ts_tablet_manager.cc:1398] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.034s user 0.002s sys 0.002s
I20240502 21:29:32.429325 12294 tablet_bootstrap.cc:492] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.433516 12251 raft_consensus.cc:2454] T 4f5655964c244669bd1b644c9399cdec P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.434466 12557 leader_election.cc:288] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.435920 12065 leader_election.cc:302] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.436575 12366 raft_consensus.cc:2781] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.436703 12366 raft_consensus.cc:686] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:32.437338 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.435379 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c09277218147c6a65b7d984a13c9b6" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.438869 12095 raft_consensus.cc:2379] T e8c09277218147c6a65b7d984a13c9b6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in this term.
I20240502 21:29:32.440663 12142 leader_election.cc:302] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.441401 12557 raft_consensus.cc:3037] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.437011 12366 consensus_queue.cc:237] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.445760 12335 consensus_queue.cc:1035] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20240502 21:29:32.447052 12452 raft_consensus.cc:504] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.448799 12452 leader_election.cc:288] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.449347 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8b568beaec444faa5e779e0b2059b1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.449528 12176 raft_consensus.cc:3037] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.450878 12557 raft_consensus.cc:2726] T e8c09277218147c6a65b7d984a13c9b6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.451997 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8b568beaec444faa5e779e0b2059b1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.452245 12098 raft_consensus.cc:3037] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.463478 12176 raft_consensus.cc:2454] T 9b8b568beaec444faa5e779e0b2059b1 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.457437 12295 tablet_bootstrap.cc:492] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.464236 12219 leader_election.cc:302] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.464500 12452 raft_consensus.cc:2781] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.464625 12452 raft_consensus.cc:686] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.464919 12452 consensus_queue.cc:237] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.466699 12295 ts_tablet_manager.cc:1367] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.016s
I20240502 21:29:32.473464 12297 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.473820 12297 ts_tablet_manager.cc:1367] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.183s user 0.040s sys 0.049s
I20240502 21:29:32.480321 12098 raft_consensus.cc:2454] T 9b8b568beaec444faa5e779e0b2059b1 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.482436 12294 tablet_bootstrap.cc:654] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.485510 11994 catalog_manager.cc:5484] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.485988 12297 raft_consensus.cc:348] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.486340 12297 raft_consensus.cc:374] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.486421 12297 raft_consensus.cc:729] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.486713 12297 consensus_queue.cc:260] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.487530 12297 ts_tablet_manager.cc:1398] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.013s user 0.000s sys 0.000s
I20240502 21:29:32.487921 12297 tablet_bootstrap.cc:492] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.490474 12297 tablet_bootstrap.cc:654] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.492372 12557 raft_consensus.cc:482] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.492529 12557 raft_consensus.cc:504] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.493350 12557 leader_election.cc:288] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.493896 12367 raft_consensus.cc:482] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.494087 12297 tablet_bootstrap.cc:492] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.494045 12367 raft_consensus.cc:504] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.494316 12297 ts_tablet_manager.cc:1367] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20240502 21:29:32.494647 12367 leader_election.cc:288] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.495250 12297 raft_consensus.cc:348] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.495535 12297 raft_consensus.cc:374] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.495654 12297 raft_consensus.cc:729] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.495817 12452 consensus_queue.cc:1035] T d4fbdb33d94f4dd196663efa7f5b1cd7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.495914 12297 consensus_queue.cc:260] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.496480 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d1079e722d4ca0b4db86807f9d7a7b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.496670 12297 ts_tablet_manager.cc:1398] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:32.496929 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e5a3ff13f40629080ec7036eb39ef" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.496996 12297 tablet_bootstrap.cc:492] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.497670 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d1079e722d4ca0b4db86807f9d7a7b" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.498035 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e5a3ff13f40629080ec7036eb39ef" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.491840 11993 catalog_manager.cc:5484] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
W20240502 21:29:32.498498 12142 leader_election.cc:341] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:32.498889 12066 leader_election.cc:341] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.498200 12095 raft_consensus.cc:2454] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
W20240502 21:29:32.500385 12065 leader_election.cc:341] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.500459 12140 leader_election.cc:302] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters: 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.500504 12065 leader_election.cc:302] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.500878 12557 raft_consensus.cc:2781] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.501005 12557 raft_consensus.cc:482] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.500857 12295 raft_consensus.cc:348] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.501068 12557 raft_consensus.cc:3037] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.501137 12295 raft_consensus.cc:374] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.501399 12295 raft_consensus.cc:729] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.501750 12295 consensus_queue.cc:260] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.502511 12295 ts_tablet_manager.cc:1398] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.036s user 0.002s sys 0.000s
I20240502 21:29:32.502862 12295 tablet_bootstrap.cc:492] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.503535 12367 raft_consensus.cc:2726] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.505020 12297 tablet_bootstrap.cc:654] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.505246 12295 tablet_bootstrap.cc:654] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.508425 12294 tablet_bootstrap.cc:492] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.508613 12294 ts_tablet_manager.cc:1367] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.079s user 0.018s sys 0.010s
I20240502 21:29:32.509609 12294 raft_consensus.cc:348] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.509902 12294 raft_consensus.cc:374] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.509976 12294 raft_consensus.cc:729] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.510293 12294 consensus_queue.cc:260] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.510769 12557 raft_consensus.cc:504] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.514501 12295 tablet_bootstrap.cc:492] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.516796 12295 ts_tablet_manager.cc:1367] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20240502 21:29:32.515905 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e5a3ff13f40629080ec7036eb39ef" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.517103 12099 raft_consensus.cc:3037] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.517706 12297 tablet_bootstrap.cc:492] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.517644 12295 raft_consensus.cc:348] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.515995 12557 leader_election.cc:288] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.517922 12295 raft_consensus.cc:374] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.517987 12295 raft_consensus.cc:729] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.515661 12294 ts_tablet_manager.cc:1398] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20240502 21:29:32.518213 12295 consensus_queue.cc:260] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.516400 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e5a3ff13f40629080ec7036eb39ef" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.518811 12295 ts_tablet_manager.cc:1398] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:32.519275 12295 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.520073 12099 raft_consensus.cc:2454] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.520630 12140 leader_election.cc:302] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.520920 12557 raft_consensus.cc:2781] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.518388 12294 tablet_bootstrap.cc:492] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.521111 12557 raft_consensus.cc:686] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.521349 12557 consensus_queue.cc:237] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
W20240502 21:29:32.521466 12142 leader_election.cc:341] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.521919 12367 raft_consensus.cc:482] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.522044 12367 raft_consensus.cc:504] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.522713 12367 leader_election.cc:288] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.522840 12294 tablet_bootstrap.cc:654] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.523386 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969594f6d4b248758a81cc2b1b5bccca" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.523577 12252 raft_consensus.cc:2454] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.517887 12297 ts_tablet_manager.cc:1367] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.012s
I20240502 21:29:32.524273 12065 leader_election.cc:302] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.524307 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969594f6d4b248758a81cc2b1b5bccca" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.524624 12367 raft_consensus.cc:2781] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.524734 12367 raft_consensus.cc:482] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.524864 12367 raft_consensus.cc:3037] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:32.524919 12066 leader_election.cc:341] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.524819 12295 tablet_bootstrap.cc:654] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.525322 12297 raft_consensus.cc:348] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.525573 12297 raft_consensus.cc:374] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.525674 12297 raft_consensus.cc:729] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.525892 12297 consensus_queue.cc:260] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.526475 12297 ts_tablet_manager.cc:1398] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 21:29:32.526983 12297 tablet_bootstrap.cc:492] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.528007 12294 tablet_bootstrap.cc:492] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.527820 12367 raft_consensus.cc:504] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.528224 12294 ts_tablet_manager.cc:1367] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20240502 21:29:32.528898 12297 tablet_bootstrap.cc:654] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.529039 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969594f6d4b248758a81cc2b1b5bccca" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.529063 12294 raft_consensus.cc:348] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.529340 12294 raft_consensus.cc:374] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.529438 12294 raft_consensus.cc:729] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
W20240502 21:29:32.529635 12066 leader_election.cc:341] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.529745 12294 consensus_queue.cc:260] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.530594 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969594f6d4b248758a81cc2b1b5bccca" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.530915 12252 raft_consensus.cc:3037] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.531492 12294 ts_tablet_manager.cc:1398] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:32.531837 12294 tablet_bootstrap.cc:492] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.533746 12294 tablet_bootstrap.cc:654] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.537351 12252 raft_consensus.cc:2454] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.537843 12367 leader_election.cc:288] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.544466 12065 leader_election.cc:302] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.548290 11996 catalog_manager.cc:5484] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.549176 12367 raft_consensus.cc:2781] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.549952 12367 raft_consensus.cc:686] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:32.550344 12367 consensus_queue.cc:237] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.551738 12295 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.552294 12295 ts_tablet_manager.cc:1367] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.004s
I20240502 21:29:32.552896 12294 tablet_bootstrap.cc:492] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.553110 12294 ts_tablet_manager.cc:1367] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.011s
I20240502 21:29:32.553398 12295 raft_consensus.cc:348] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.553632 12295 raft_consensus.cc:374] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.553740 12295 raft_consensus.cc:729] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.553987 12295 consensus_queue.cc:260] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.554105 12294 raft_consensus.cc:348] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.554366 12294 raft_consensus.cc:374] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.554476 12294 raft_consensus.cc:729] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.554728 12295 ts_tablet_manager.cc:1398] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.554953 12297 tablet_bootstrap.cc:492] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.555140 12297 ts_tablet_manager.cc:1367] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.015s
I20240502 21:29:32.555114 12295 tablet_bootstrap.cc:492] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.554745 12294 consensus_queue.cc:260] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.555887 12297 raft_consensus.cc:348] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.556241 12297 raft_consensus.cc:374] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.556349 12297 raft_consensus.cc:729] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.556643 12294 ts_tablet_manager.cc:1398] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:32.557010 12294 tablet_bootstrap.cc:492] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.557289 12295 tablet_bootstrap.cc:654] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.557646 11996 catalog_manager.cc:5484] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.559024 12294 tablet_bootstrap.cc:654] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.559257 12297 consensus_queue.cc:260] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.561928 12297 ts_tablet_manager.cc:1398] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20240502 21:29:32.562896 12297 tablet_bootstrap.cc:492] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.564714 12297 tablet_bootstrap.cc:654] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.564932 12294 tablet_bootstrap.cc:492] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.565140 12294 ts_tablet_manager.cc:1367] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20240502 21:29:32.566120 12294 raft_consensus.cc:348] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.566486 12294 raft_consensus.cc:374] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.566598 12294 raft_consensus.cc:729] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.566938 12294 consensus_queue.cc:260] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.567847 12294 ts_tablet_manager.cc:1398] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240502 21:29:32.568320 12294 tablet_bootstrap.cc:492] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.570298 12295 tablet_bootstrap.cc:492] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.570428 12294 tablet_bootstrap.cc:654] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.570554 12295 ts_tablet_manager.cc:1367] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20240502 21:29:32.571399 12295 raft_consensus.cc:348] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.571754 12295 raft_consensus.cc:374] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.571871 12295 raft_consensus.cc:729] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.572250 12295 consensus_queue.cc:260] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.575124 12295 ts_tablet_manager.cc:1398] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:32.575629 12295 tablet_bootstrap.cc:492] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.578172 12295 tablet_bootstrap.cc:654] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.579691 12557 raft_consensus.cc:482] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.579871 12557 raft_consensus.cc:504] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.580652 12557 leader_election.cc:288] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.581643 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb074bc2833742dda1e8e68ca5f2c8e7" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.581840 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb074bc2833742dda1e8e68ca5f2c8e7" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.581921 12099 raft_consensus.cc:2454] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.582049 12252 raft_consensus.cc:2454] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.582489 12140 leader_election.cc:302] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.582780 12557 raft_consensus.cc:2781] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.582918 12557 raft_consensus.cc:482] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.583015 12557 raft_consensus.cc:3037] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.585647 12557 raft_consensus.cc:504] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.586722 12297 tablet_bootstrap.cc:492] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.586913 12297 ts_tablet_manager.cc:1367] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.008s
I20240502 21:29:32.587111 12294 tablet_bootstrap.cc:492] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.587273 12294 ts_tablet_manager.cc:1367] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.011s
I20240502 21:29:32.588021 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb074bc2833742dda1e8e68ca5f2c8e7" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.588277 12099 raft_consensus.cc:3037] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.588794 12557 leader_election.cc:288] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.590709 12297 raft_consensus.cc:348] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.590867 12557 consensus_queue.cc:1035] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240502 21:29:32.590939 12297 raft_consensus.cc:374] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.591142 12297 raft_consensus.cc:729] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.590813 12294 raft_consensus.cc:348] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.591378 12294 raft_consensus.cc:374] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.591439 12099 raft_consensus.cc:2454] T fb074bc2833742dda1e8e68ca5f2c8e7 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.591501 12294 raft_consensus.cc:729] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.591676 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb074bc2833742dda1e8e68ca5f2c8e7" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.591758 12294 consensus_queue.cc:260] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.591910 12252 raft_consensus.cc:3037] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.592101 12140 leader_election.cc:302] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.592432 12294 ts_tablet_manager.cc:1398] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240502 21:29:32.592782 12294 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.593856 12557 raft_consensus.cc:2781] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.594072 12557 raft_consensus.cc:686] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.594391 12557 consensus_queue.cc:237] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.594798 12294 tablet_bootstrap.cc:654] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.595146 12252 raft_consensus.cc:2454] T fb074bc2833742dda1e8e68ca5f2c8e7 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.595443 12295 tablet_bootstrap.cc:492] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.595618 12295 ts_tablet_manager.cc:1367] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20240502 21:29:32.595897 12297 consensus_queue.cc:260] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.596545 12297 ts_tablet_manager.cc:1398] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20240502 21:29:32.596472 12295 raft_consensus.cc:348] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.596691 12295 raft_consensus.cc:374] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.596799 12295 raft_consensus.cc:729] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.597132 12297 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.597013 12295 consensus_queue.cc:260] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.597637 12295 ts_tablet_manager.cc:1398] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:32.597972 12295 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.599176 12297 tablet_bootstrap.cc:654] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.602051 12448 consensus_queue.cc:1035] T 17e15a0d7054443ea55550d20ee0518e P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.603734 12295 tablet_bootstrap.cc:654] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.607592 12294 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.607765 12294 ts_tablet_manager.cc:1367] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20240502 21:29:32.608448 12294 raft_consensus.cc:348] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.608667 12294 raft_consensus.cc:374] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.608726 12294 raft_consensus.cc:729] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.608949 12294 consensus_queue.cc:260] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.609555 12294 ts_tablet_manager.cc:1398] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:32.610284 12294 tablet_bootstrap.cc:492] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0: Bootstrap starting.
I20240502 21:29:32.612295 12294 tablet_bootstrap.cc:654] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.615363 12367 raft_consensus.cc:482] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.615603 12367 raft_consensus.cc:504] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.616750 12367 leader_election.cc:288] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.617360 12366 raft_consensus.cc:482] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.617576 12366 raft_consensus.cc:504] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.618438 12366 leader_election.cc:288] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.619508 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158108cfdab4223b8f9d0226578de60" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.620067 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.621613 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158108cfdab4223b8f9d0226578de60" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.622083 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
W20240502 21:29:32.622947 12065 leader_election.cc:341] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:32.623287 12065 leader_election.cc:341] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:32.623742 12066 leader_election.cc:341] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.623973 12066 leader_election.cc:302] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
W20240502 21:29:32.624699 12066 leader_election.cc:341] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.624886 12066 leader_election.cc:302] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.625203 12316 raft_consensus.cc:482] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.625430 12316 raft_consensus.cc:504] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.626410 12316 leader_election.cc:288] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.626642 12316 raft_consensus.cc:2726] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.626850 12316 raft_consensus.cc:2726] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.628147 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df964090d307434981236aa967944cee" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.629357 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df964090d307434981236aa967944cee" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
W20240502 21:29:32.630167 12065 leader_election.cc:341] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
W20240502 21:29:32.630645 12066 leader_election.cc:341] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.630867 12066 leader_election.cc:302] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.631708 12366 raft_consensus.cc:2726] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.633579 12448 consensus_queue.cc:1035] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.636459 11996 catalog_manager.cc:5484] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.639093 12297 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.639297 12297 ts_tablet_manager.cc:1367] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.012s
I20240502 21:29:32.639683 12295 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.639843 12295 ts_tablet_manager.cc:1367] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.042s user 0.010s sys 0.007s
I20240502 21:29:32.640187 12297 raft_consensus.cc:348] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.640429 12297 raft_consensus.cc:374] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.640517 12297 raft_consensus.cc:729] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.640661 12363 consensus_queue.cc:1035] T 73d4e5a4db934406b9f995442ec6bcd1 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.640766 12297 consensus_queue.cc:260] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.641484 12297 ts_tablet_manager.cc:1398] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:32.641525 12316 raft_consensus.cc:482] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.641822 12297 tablet_bootstrap.cc:492] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.641728 12316 raft_consensus.cc:504] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.643033 12316 leader_election.cc:288] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.643527 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.644068 12297 tablet_bootstrap.cc:654] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
W20240502 21:29:32.644245 12065 leader_election.cc:341] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.646093 12316 raft_consensus.cc:482] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.646338 12316 raft_consensus.cc:504] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.647559 12316 leader_election.cc:288] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.647979 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.649384 12363 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.652865 12295 raft_consensus.cc:348] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.653154 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a729e66312d4e7fb61c9f7ecfe8d271" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.653399 12251 raft_consensus.cc:2454] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.653167 12295 raft_consensus.cc:374] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.653564 12295 raft_consensus.cc:729] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.653860 12295 consensus_queue.cc:260] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.654762 12295 ts_tablet_manager.cc:1398] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20240502 21:29:32.655118 12295 tablet_bootstrap.cc:492] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.656981 12065 leader_election.cc:302] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.657688 12295 tablet_bootstrap.cc:654] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.660343 12363 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.661104 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a729e66312d4e7fb61c9f7ecfe8d271" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.661362 12174 raft_consensus.cc:2454] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.661883 12295 tablet_bootstrap.cc:492] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.662173 12295 ts_tablet_manager.cc:1367] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240502 21:29:32.668357 12316 raft_consensus.cc:2781] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.668840 12316 raft_consensus.cc:482] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.669109 12316 raft_consensus.cc:3037] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.670173 12297 tablet_bootstrap.cc:492] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.670439 12297 ts_tablet_manager.cc:1367] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.019s
I20240502 21:29:32.671386 12297 raft_consensus.cc:348] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.671676 12297 raft_consensus.cc:374] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.671782 12297 raft_consensus.cc:729] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.672151 12297 consensus_queue.cc:260] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.673768 12295 raft_consensus.cc:348] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.673967 12295 raft_consensus.cc:374] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.674046 12295 raft_consensus.cc:729] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.674257 12295 consensus_queue.cc:260] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.675202 12297 ts_tablet_manager.cc:1398] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240502 21:29:32.675362 12361 raft_consensus.cc:482] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.675496 12361 raft_consensus.cc:504] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.676146 12361 leader_election.cc:288] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.676815 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969594f6d4b248758a81cc2b1b5bccca" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.677659 12297 tablet_bootstrap.cc:492] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.679461 12316 raft_consensus.cc:504] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.681317 12297 tablet_bootstrap.cc:654] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.683358 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "969594f6d4b248758a81cc2b1b5bccca" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.683892 12252 raft_consensus.cc:2379] T 969594f6d4b248758a81cc2b1b5bccca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:32.684893 12142 leader_election.cc:302] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.685456 12361 raft_consensus.cc:3037] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.689900 12361 raft_consensus.cc:2726] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.691110 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a729e66312d4e7fb61c9f7ecfe8d271" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.691403 12174 raft_consensus.cc:3037] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.694875 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a729e66312d4e7fb61c9f7ecfe8d271" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.695154 12253 raft_consensus.cc:3037] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.695688 12174 raft_consensus.cc:2454] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.696622 12297 tablet_bootstrap.cc:492] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.696936 12297 ts_tablet_manager.cc:1367] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20240502 21:29:32.697146 12066 leader_election.cc:302] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.697839 12297 raft_consensus.cc:348] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.698083 12297 raft_consensus.cc:374] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.698150 12297 raft_consensus.cc:729] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.697971 12309 raft_consensus.cc:2781] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.698338 12309 raft_consensus.cc:686] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:32.698386 12297 consensus_queue.cc:260] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.698680 12309 consensus_queue.cc:237] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.699126 12297 ts_tablet_manager.cc:1398] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.699506 12295 ts_tablet_manager.cc:1398] T 969594f6d4b248758a81cc2b1b5bccca P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.037s user 0.001s sys 0.001s
I20240502 21:29:32.699863 12295 tablet_bootstrap.cc:492] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.700901 12297 tablet_bootstrap.cc:492] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.702867 12297 tablet_bootstrap.cc:654] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.703922 12452 consensus_queue.cc:1035] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.705488 12253 raft_consensus.cc:2454] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.708189 12295 tablet_bootstrap.cc:654] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.714926 12557 raft_consensus.cc:482] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.715158 12557 raft_consensus.cc:504] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.716485 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a7f75fa74945c09343353c01f4b31b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.716770 12098 raft_consensus.cc:2454] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.717305 12140 leader_election.cc:302] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.717726 12361 raft_consensus.cc:2781] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.717832 12361 raft_consensus.cc:482] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.717938 12361 raft_consensus.cc:3037] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.718950 12295 tablet_bootstrap.cc:492] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.719828 12295 ts_tablet_manager.cc:1367] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.001s
I20240502 21:29:32.721312 12295 raft_consensus.cc:348] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.721733 12295 raft_consensus.cc:374] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.721872 12295 raft_consensus.cc:729] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.722018 12452 consensus_queue.cc:1035] T e8c09277218147c6a65b7d984a13c9b6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.722177 12295 consensus_queue.cc:260] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.723222 12295 ts_tablet_manager.cc:1398] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240502 21:29:32.723613 12295 tablet_bootstrap.cc:492] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.725392 12361 raft_consensus.cc:504] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.726934 12295 tablet_bootstrap.cc:654] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.727108 12297 tablet_bootstrap.cc:492] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.727258 12297 ts_tablet_manager.cc:1367] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.015s
I20240502 21:29:32.728022 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a7f75fa74945c09343353c01f4b31b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.728291 12096 raft_consensus.cc:3037] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.728286 12297 raft_consensus.cc:348] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.728561 12297 raft_consensus.cc:374] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.728631 12297 raft_consensus.cc:729] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.728895 12297 consensus_queue.cc:260] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.731406 12096 raft_consensus.cc:2454] T 04a7f75fa74945c09343353c01f4b31b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.732196 12297 ts_tablet_manager.cc:1398] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240502 21:29:32.732188 12140 leader_election.cc:302] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.732668 12297 tablet_bootstrap.cc:492] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.732906 12556 raft_consensus.cc:2781] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.733054 12556 raft_consensus.cc:686] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.733361 12556 consensus_queue.cc:237] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.735019 12297 tablet_bootstrap.cc:654] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.737294 12366 raft_consensus.cc:482] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.737473 12366 raft_consensus.cc:504] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.738303 12366 leader_election.cc:288] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.739235 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a160b9e9063a4b95938cf7a5c944d77a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.739493 12251 raft_consensus.cc:2454] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.740108 12065 leader_election.cc:302] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.740514 12366 raft_consensus.cc:2781] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.740608 12366 raft_consensus.cc:482] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.740689 12366 raft_consensus.cc:3037] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.743566 12297 tablet_bootstrap.cc:492] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.743566 12295 tablet_bootstrap.cc:492] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.743736 12297 ts_tablet_manager.cc:1367] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20240502 21:29:32.743736 12295 ts_tablet_manager.cc:1367] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.003s
I20240502 21:29:32.744679 12297 raft_consensus.cc:348] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.744679 12295 raft_consensus.cc:348] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.744931 12297 raft_consensus.cc:374] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.744932 12295 raft_consensus.cc:374] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.745013 12297 raft_consensus.cc:729] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.745021 12295 raft_consensus.cc:729] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.745254 12295 consensus_queue.cc:260] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.744732 12366 raft_consensus.cc:504] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.745415 12557 leader_election.cc:288] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.745888 12295 ts_tablet_manager.cc:1398] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:32.745254 12297 consensus_queue.cc:260] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.746814 12297 ts_tablet_manager.cc:1398] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 21:29:32.747416 12366 leader_election.cc:288] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.747648 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a160b9e9063a4b95938cf7a5c944d77a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.747812 12253 raft_consensus.cc:3037] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.748461 12297 tablet_bootstrap.cc:492] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.750481 12297 tablet_bootstrap.cc:654] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.750984 12253 raft_consensus.cc:2454] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.751489 12065 leader_election.cc:302] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.751888 12366 raft_consensus.cc:2781] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.752178 12366 raft_consensus.cc:686] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:32.752665 12366 consensus_queue.cc:237] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.746217 12361 leader_election.cc:288] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.758823 12556 consensus_queue.cc:1035] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.760324 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a7f75fa74945c09343353c01f4b31b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.760576 12252 raft_consensus.cc:3037] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.761410 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a7f75fa74945c09343353c01f4b31b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.764590 12252 raft_consensus.cc:2454] T 04a7f75fa74945c09343353c01f4b31b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.765514 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a160b9e9063a4b95938cf7a5c944d77a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.766448 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a160b9e9063a4b95938cf7a5c944d77a" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.767192 12297 tablet_bootstrap.cc:492] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.767333 12297 ts_tablet_manager.cc:1367] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.012s
I20240502 21:29:32.746218 12295 tablet_bootstrap.cc:492] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.768263 12297 raft_consensus.cc:348] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.768476 12297 raft_consensus.cc:374] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.768535 12297 raft_consensus.cc:729] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.768798 12297 consensus_queue.cc:260] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.769368 12297 ts_tablet_manager.cc:1398] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.769688 12297 tablet_bootstrap.cc:492] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.770378 12316 leader_election.cc:288] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.771574 12297 tablet_bootstrap.cc:654] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.776540 12295 tablet_bootstrap.cc:654] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.784881 12557 consensus_queue.cc:1035] T 528a557fdd90485d91077d2a61095878 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.778532 12297 tablet_bootstrap.cc:492] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.778882 12361 raft_consensus.cc:482] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240502 21:29:32.779306 12066 leader_election.cc:341] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.785442 12361 raft_consensus.cc:504] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
W20240502 21:29:32.785610 12066 leader_election.cc:341] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.786665 12361 leader_election.cc:288] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.783779 11994 catalog_manager.cc:5484] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.786937 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8307fd623a499fa05cd9eca238ddc2" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.787282 12099 raft_consensus.cc:2454] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.787632 12140 leader_election.cc:302] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.787561 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8307fd623a499fa05cd9eca238ddc2" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.788190 12253 raft_consensus.cc:2454] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.788214 12297 ts_tablet_manager.cc:1367] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.002s
I20240502 21:29:32.787993 12361 raft_consensus.cc:2781] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.788810 12361 raft_consensus.cc:482] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.789033 12361 raft_consensus.cc:3037] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.789355 12297 raft_consensus.cc:348] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.789603 12297 raft_consensus.cc:374] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.789708 12297 raft_consensus.cc:729] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.791406 12297 consensus_queue.cc:260] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.792076 12297 ts_tablet_manager.cc:1398] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 21:29:32.793835 12297 tablet_bootstrap.cc:492] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.796838 12297 tablet_bootstrap.cc:654] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.798601 11994 catalog_manager.cc:5484] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.799333 11994 catalog_manager.cc:5484] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.803715 12361 raft_consensus.cc:504] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.804705 12361 leader_election.cc:288] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.804977 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8307fd623a499fa05cd9eca238ddc2" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.805166 12095 raft_consensus.cc:3037] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.805351 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8307fd623a499fa05cd9eca238ddc2" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.805510 12252 raft_consensus.cc:3037] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.807760 12309 raft_consensus.cc:482] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.807997 12095 raft_consensus.cc:2454] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.807945 12309 raft_consensus.cc:504] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.808336 12252 raft_consensus.cc:2454] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.808390 12140 leader_election.cc:302] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.809289 12556 raft_consensus.cc:2781] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.809448 12556 raft_consensus.cc:686] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.809424 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc5520fead4142819e3842fd573287" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.809603 12174 raft_consensus.cc:2454] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.809710 12556 consensus_queue.cc:237] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.809948 12066 leader_election.cc:302] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.810875 12294 tablet_bootstrap.cc:492] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0: No bootstrap required, opened a new log
I20240502 21:29:32.811033 12294 ts_tablet_manager.cc:1367] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0: Time spent bootstrapping tablet: real 0.201s user 0.036s sys 0.085s
W20240502 21:29:32.812171 12066 leader_election.cc:341] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.812291 12066 leader_election.cc:302] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0; no voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.812536 12297 tablet_bootstrap.cc:492] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.812655 12297 ts_tablet_manager.cc:1367] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20240502 21:29:32.812955 12367 raft_consensus.cc:2781] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.813050 12367 raft_consensus.cc:482] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.813117 12367 raft_consensus.cc:3037] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.813511 12297 raft_consensus.cc:348] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.813710 12297 raft_consensus.cc:374] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.813759 12297 raft_consensus.cc:729] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.813964 12297 consensus_queue.cc:260] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.814522 12297 ts_tablet_manager.cc:1398] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:32.814810 12297 tablet_bootstrap.cc:492] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.815692 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc5520fead4142819e3842fd573287" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.815938 12309 leader_election.cc:288] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.816104 12309 raft_consensus.cc:2726] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.811913 12294 raft_consensus.cc:348] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.816833 12294 raft_consensus.cc:374] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.816910 12294 raft_consensus.cc:729] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.815806 12367 raft_consensus.cc:504] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.821007 12367 leader_election.cc:288] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.815876 12252 raft_consensus.cc:2454] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.821635 12294 consensus_queue.cc:260] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.821986 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc5520fead4142819e3842fd573287" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.822185 12253 raft_consensus.cc:3037] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.822691 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc5520fead4142819e3842fd573287" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.822849 12176 raft_consensus.cc:3037] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.824681 12294 ts_tablet_manager.cc:1398] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20240502 21:29:32.826684 12297 tablet_bootstrap.cc:654] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.827235 12253 raft_consensus.cc:2454] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.827649 12065 leader_election.cc:302] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.827898 12367 raft_consensus.cc:2781] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.828081 12367 raft_consensus.cc:686] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:32.828667 12295 tablet_bootstrap.cc:492] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.828801 12295 ts_tablet_manager.cc:1367] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.083s user 0.003s sys 0.000s
I20240502 21:29:32.829967 12176 raft_consensus.cc:2454] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.828344 12367 consensus_queue.cc:237] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.831889 12295 raft_consensus.cc:348] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.832150 12295 raft_consensus.cc:374] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.832212 12295 raft_consensus.cc:729] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.832402 12295 consensus_queue.cc:260] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.832873 12295 ts_tablet_manager.cc:1398] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:32.833144 12295 tablet_bootstrap.cc:492] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.835659 12335 raft_consensus.cc:482] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.835851 12335 raft_consensus.cc:504] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.836771 12335 leader_election.cc:288] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.837564 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6c003ea2884ca9a1827814a4930742" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.837832 12099 raft_consensus.cc:2454] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.838384 12216 leader_election.cc:302] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.838889 12335 raft_consensus.cc:2781] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.838972 12335 raft_consensus.cc:482] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.839031 12335 raft_consensus.cc:3037] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.841634 12335 raft_consensus.cc:504] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.843242 11994 catalog_manager.cc:5484] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.843945 12335 leader_election.cc:288] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.844462 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6c003ea2884ca9a1827814a4930742" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.844650 12174 raft_consensus.cc:3037] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.845853 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6c003ea2884ca9a1827814a4930742" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.846192 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6c003ea2884ca9a1827814a4930742" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.846343 12099 raft_consensus.cc:3037] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.848596 12174 raft_consensus.cc:2454] T dd6c003ea2884ca9a1827814a4930742 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.849042 12219 leader_election.cc:302] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.849434 12335 raft_consensus.cc:2781] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.849598 12335 raft_consensus.cc:686] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.849908 12335 consensus_queue.cc:237] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.850895 12295 tablet_bootstrap.cc:654] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.851111 12099 raft_consensus.cc:2454] T dd6c003ea2884ca9a1827814a4930742 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.856674 11996 catalog_manager.cc:5484] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.860535 12297 tablet_bootstrap.cc:492] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.860679 12295 tablet_bootstrap.cc:492] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.860863 12295 ts_tablet_manager.cc:1367] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20240502 21:29:32.861064 12297 ts_tablet_manager.cc:1367] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.003s
I20240502 21:29:32.864238 12295 raft_consensus.cc:348] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.865767 12295 raft_consensus.cc:374] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.865998 12295 raft_consensus.cc:729] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.866340 12295 consensus_queue.cc:260] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.866963 12297 raft_consensus.cc:348] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.867233 12295 ts_tablet_manager.cc:1398] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240502 21:29:32.867267 12297 raft_consensus.cc:374] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.867381 12297 raft_consensus.cc:729] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.867591 12295 tablet_bootstrap.cc:492] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.867565 12335 raft_consensus.cc:482] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.867815 12297 consensus_queue.cc:260] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.867842 12335 raft_consensus.cc:504] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.868461 12297 ts_tablet_manager.cc:1398] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20240502 21:29:32.868955 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.869344 12099 raft_consensus.cc:2454] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.869113 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.868626 12335 leader_election.cc:288] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.869939 11994 catalog_manager.cc:5484] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.869884 12216 leader_election.cc:302] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
W20240502 21:29:32.871520 12219 leader_election.cc:341] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.871641 12454 raft_consensus.cc:2781] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.871757 12454 raft_consensus.cc:482] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.871876 12454 raft_consensus.cc:3037] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.869881 12297 tablet_bootstrap.cc:492] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.874796 12454 raft_consensus.cc:504] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.875062 12297 tablet_bootstrap.cc:654] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.875919 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.876183 12099 raft_consensus.cc:3037] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.876142 12335 consensus_queue.cc:1035] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.876935 12295 tablet_bootstrap.cc:654] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.879084 12099 raft_consensus.cc:2454] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.879633 12454 leader_election.cc:288] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.880784 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.880919 12297 tablet_bootstrap.cc:492] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.881084 12297 ts_tablet_manager.cc:1367] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
W20240502 21:29:32.881700 12219 leader_election.cc:341] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.881999 12216 leader_election.cc:302] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.882719 12334 raft_consensus.cc:2781] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.882877 12334 raft_consensus.cc:686] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.883200 12334 consensus_queue.cc:237] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.883814 12297 raft_consensus.cc:348] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.884054 12297 raft_consensus.cc:374] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.884250 12297 raft_consensus.cc:729] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.884557 12297 consensus_queue.cc:260] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.885250 12297 ts_tablet_manager.cc:1398] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 21:29:32.885485 12295 tablet_bootstrap.cc:492] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.885596 12297 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: Bootstrap starting.
I20240502 21:29:32.885668 12295 ts_tablet_manager.cc:1367] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.001s
I20240502 21:29:32.885908 12452 consensus_queue.cc:1035] T 834684d85d644236aab5229b549c428a P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.887419 12297 tablet_bootstrap.cc:654] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.888078 12367 consensus_queue.cc:1035] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.893743 12297 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: No bootstrap required, opened a new log
I20240502 21:29:32.894023 12297 ts_tablet_manager.cc:1367] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20240502 21:29:32.895157 12297 raft_consensus.cc:348] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.895500 12297 raft_consensus.cc:374] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.895607 12297 raft_consensus.cc:729] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.895854 12297 consensus_queue.cc:260] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.896658 12297 ts_tablet_manager.cc:1398] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:32.901051 12295 raft_consensus.cc:348] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.901407 12295 raft_consensus.cc:374] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.901528 12295 raft_consensus.cc:729] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.901863 12295 consensus_queue.cc:260] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.902685 12295 ts_tablet_manager.cc:1398] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20240502 21:29:32.904351 12295 tablet_bootstrap.cc:492] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.907723 11994 catalog_manager.cc:5484] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.909790 12174 raft_consensus.cc:3037] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.913046 12295 tablet_bootstrap.cc:654] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.913463 12367 consensus_queue.cc:1035] T 4f5655964c244669bd1b644c9399cdec P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:32.913956 12361 raft_consensus.cc:482] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.914116 12361 raft_consensus.cc:504] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.914952 12361 leader_election.cc:288] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.915390 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5655964c244669bd1b644c9399cdec" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.915688 12295 tablet_bootstrap.cc:492] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.915913 12295 ts_tablet_manager.cc:1367] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.001s
I20240502 21:29:32.916683 12295 raft_consensus.cc:348] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.916855 12295 raft_consensus.cc:374] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.916970 12295 raft_consensus.cc:729] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.917207 12295 consensus_queue.cc:260] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.917935 12334 raft_consensus.cc:482] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.917949 12295 ts_tablet_manager.cc:1398] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:32.918221 12334 raft_consensus.cc:504] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.918460 12295 tablet_bootstrap.cc:492] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.919430 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.919512 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5655964c244669bd1b644c9399cdec" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.920069 12142 leader_election.cc:302] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.920607 12295 tablet_bootstrap.cc:654] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
W20240502 21:29:32.920581 12219 leader_election.cc:341] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.919446 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.921046 12099 raft_consensus.cc:2454] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.921213 12334 leader_election.cc:288] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.921084 12361 raft_consensus.cc:2726] T 4f5655964c244669bd1b644c9399cdec P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 21:29:32.921658 12216 leader_election.cc:302] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.922097 12334 raft_consensus.cc:2781] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.922266 12334 raft_consensus.cc:482] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.922470 12334 raft_consensus.cc:3037] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.927786 12334 raft_consensus.cc:504] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.929131 12334 leader_election.cc:288] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.929529 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.929677 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.929776 12095 raft_consensus.cc:3037] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
W20240502 21:29:32.930284 12219 leader_election.cc:341] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): Illegal state: must be running to vote when last-logged opid is not known
I20240502 21:29:32.932150 12295 tablet_bootstrap.cc:492] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.932363 12295 ts_tablet_manager.cc:1367] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20240502 21:29:32.932360 12095 raft_consensus.cc:2454] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.932967 12216 leader_election.cc:302] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters: 17f04379252d4ecfab3a4916c61a984f
I20240502 21:29:32.933241 12334 raft_consensus.cc:2781] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.933205 12295 raft_consensus.cc:348] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.933461 12295 raft_consensus.cc:374] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.933574 12334 raft_consensus.cc:686] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.933578 12295 raft_consensus.cc:729] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.933836 12295 consensus_queue.cc:260] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.933869 12334 consensus_queue.cc:237] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.934430 12295 ts_tablet_manager.cc:1398] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.935041 12295 tablet_bootstrap.cc:492] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.936851 12295 tablet_bootstrap.cc:654] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.938844 12295 tablet_bootstrap.cc:492] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.939038 12295 ts_tablet_manager.cc:1367] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:32.939244 11993 catalog_manager.cc:5484] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.939882 12295 raft_consensus.cc:348] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.940169 12295 raft_consensus.cc:374] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.940289 12295 raft_consensus.cc:729] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.940550 12295 consensus_queue.cc:260] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.941166 12295 ts_tablet_manager.cc:1398] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 21:29:32.941505 12295 tablet_bootstrap.cc:492] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.943256 12295 tablet_bootstrap.cc:654] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.947590 12295 tablet_bootstrap.cc:492] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.947767 12295 ts_tablet_manager.cc:1367] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20240502 21:29:32.948446 12295 raft_consensus.cc:348] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.948639 12295 raft_consensus.cc:374] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.948724 12295 raft_consensus.cc:729] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.948938 12295 consensus_queue.cc:260] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.949378 12295 ts_tablet_manager.cc:1398] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:32.949759 12295 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: Bootstrap starting.
I20240502 21:29:32.951297 12295 tablet_bootstrap.cc:654] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: Neither blocks nor log segments found. Creating new log.
I20240502 21:29:32.953336 12295 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: No bootstrap required, opened a new log
I20240502 21:29:32.953511 12295 ts_tablet_manager.cc:1367] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 21:29:32.954360 12295 raft_consensus.cc:348] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.954597 12295 raft_consensus.cc:374] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 21:29:32.954720 12295 raft_consensus.cc:729] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Initialized, Role: FOLLOWER
I20240502 21:29:32.954958 12295 consensus_queue.cc:260] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.955623 12295 ts_tablet_manager.cc:1398] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:32.956442 12361 raft_consensus.cc:482] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.956600 12361 raft_consensus.cc:504] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.957331 12361 leader_election.cc:288] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.957810 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d1079e722d4ca0b4db86807f9d7a7b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.958045 12253 raft_consensus.cc:2454] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.958369 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d1079e722d4ca0b4db86807f9d7a7b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.958552 12099 raft_consensus.cc:2454] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:32.958961 12140 leader_election.cc:302] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.959275 12361 raft_consensus.cc:2781] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.959408 12361 raft_consensus.cc:482] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.959524 12361 raft_consensus.cc:3037] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.961869 12361 raft_consensus.cc:504] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.962250 12557 raft_consensus.cc:482] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.962401 12557 raft_consensus.cc:504] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.962551 12361 leader_election.cc:288] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.963025 12557 leader_election.cc:288] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.963591 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d1079e722d4ca0b4db86807f9d7a7b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.963680 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a160b9e9063a4b95938cf7a5c944d77a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.963762 12095 raft_consensus.cc:3037] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.964610 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d1079e722d4ca0b4db86807f9d7a7b" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.964615 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a160b9e9063a4b95938cf7a5c944d77a" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.964795 12253 raft_consensus.cc:3037] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.964854 12252 raft_consensus.cc:2379] T a160b9e9063a4b95938cf7a5c944d77a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 0672163bb07b41a1b14342a8efcb24a0 in this term.
I20240502 21:29:32.965324 12142 leader_election.cc:302] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.965705 12557 raft_consensus.cc:3037] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.966547 12095 raft_consensus.cc:2454] T 40d1079e722d4ca0b4db86807f9d7a7b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.967569 12253 raft_consensus.cc:2454] T 40d1079e722d4ca0b4db86807f9d7a7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:32.968329 12557 raft_consensus.cc:2726] T a160b9e9063a4b95938cf7a5c944d77a P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.968752 12142 leader_election.cc:302] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.968878 12334 raft_consensus.cc:482] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.969008 12334 raft_consensus.cc:504] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.969609 12557 raft_consensus.cc:2781] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.969622 12334 leader_election.cc:288] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.969786 12557 raft_consensus.cc:686] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:32.970011 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158108cfdab4223b8f9d0226578de60" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.970048 12557 consensus_queue.cc:237] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.970186 12095 raft_consensus.cc:2454] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.970644 12216 leader_election.cc:302] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.970975 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158108cfdab4223b8f9d0226578de60" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.971201 12174 raft_consensus.cc:2454] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:32.970953 12454 raft_consensus.cc:2781] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.971457 12454 raft_consensus.cc:482] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.971529 12454 raft_consensus.cc:3037] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.973285 12367 raft_consensus.cc:482] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.973425 12367 raft_consensus.cc:504] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.974081 12367 leader_election.cc:288] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.974711 12454 raft_consensus.cc:504] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.974761 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19293e7da2774ce4aa900e851931b6ae" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:32.974953 12174 raft_consensus.cc:2454] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.975325 12066 leader_election.cc:302] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:32.975580 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158108cfdab4223b8f9d0226578de60" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:32.975682 12367 raft_consensus.cc:2781] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:32.975749 12367 raft_consensus.cc:482] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:32.975760 12095 raft_consensus.cc:3037] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.975847 12367 raft_consensus.cc:3037] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.976104 12556 raft_consensus.cc:482] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:32.976228 12556 raft_consensus.cc:504] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.976730 12556 leader_election.cc:288] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.977643 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.978564 12095 raft_consensus.cc:2454] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.978626 12367 raft_consensus.cc:504] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.978946 12216 leader_election.cc:302] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.979043 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19293e7da2774ce4aa900e851931b6ae" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:32.979182 12334 raft_consensus.cc:2781] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.979214 12253 raft_consensus.cc:2454] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 0.
I20240502 21:29:32.979305 12334 raft_consensus.cc:686] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:32.979635 12334 consensus_queue.cc:237] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.979708 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19293e7da2774ce4aa900e851931b6ae" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:32.979866 12252 raft_consensus.cc:3037] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.980207 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ef70a542484291bfb51c87b4d2f838" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:32.980235 12174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19293e7da2774ce4aa900e851931b6ae" candidate_uuid: "0672163bb07b41a1b14342a8efcb24a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.980383 12174 raft_consensus.cc:3037] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.980435 12095 raft_consensus.cc:2379] T f5ef70a542484291bfb51c87b4d2f838 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in this term.
I20240502 21:29:32.980829 12140 leader_election.cc:302] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:32.981319 12301 raft_consensus.cc:3037] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.982808 12252 raft_consensus.cc:2454] T 19293e7da2774ce4aa900e851931b6ae P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.983304 12065 leader_election.cc:302] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:32.983687 12309 raft_consensus.cc:2781] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:32.983840 12309 raft_consensus.cc:686] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0672163bb07b41a1b14342a8efcb24a0, State: Running, Role: LEADER
I20240502 21:29:32.984112 12301 raft_consensus.cc:2726] T f5ef70a542484291bfb51c87b4d2f838 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:32.984123 12309 consensus_queue.cc:237] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:32.979247 12367 leader_election.cc:288] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [CANDIDATE]: Term 1 election: Requested vote from peers 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:32.986029 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b158108cfdab4223b8f9d0226578de60" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:32.986230 12176 raft_consensus.cc:3037] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:32.987182 11994 catalog_manager.cc:5484] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:32.983369 12454 leader_election.cc:288] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:32.988636 12174 raft_consensus.cc:2454] T 19293e7da2774ce4aa900e851931b6ae P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0672163bb07b41a1b14342a8efcb24a0 in term 1.
I20240502 21:29:32.989043 12176 raft_consensus.cc:2454] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:32.988981 11993 catalog_manager.cc:5484] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65). New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:32.993477 11993 catalog_manager.cc:5484] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:33.012269 12557 consensus_queue.cc:1035] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.015749 12253 raft_consensus.cc:3037] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.019354 12301 consensus_queue.cc:1035] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.040798 12557 consensus_queue.cc:1035] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.045547 12301 raft_consensus.cc:482] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:33.045734 12301 raft_consensus.cc:504] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.046407 12301 leader_election.cc:288] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:33.046558 12301 consensus_queue.cc:1035] T fb074bc2833742dda1e8e68ca5f2c8e7 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.047271 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:33.047681 12095 raft_consensus.cc:2379] T 1efbb27ede324d8d9005518cf8a66360 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17f04379252d4ecfab3a4916c61a984f in current term 1: Already voted for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in this term.
I20240502 21:29:33.047283 12253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efbb27ede324d8d9005518cf8a66360" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:33.048630 12142 leader_election.cc:302] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f; no voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9
I20240502 21:29:33.049006 12301 raft_consensus.cc:3037] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.051931 12301 raft_consensus.cc:2726] T 1efbb27ede324d8d9005518cf8a66360 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 21:29:33.055326 12454 raft_consensus.cc:482] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:33.055513 12454 raft_consensus.cc:504] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.056281 12454 leader_election.cc:288] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:33.057027 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbfbbbb346f413782d97580adfa82c1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:33.057032 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbfbbbb346f413782d97580adfa82c1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:33.057286 12095 raft_consensus.cc:2454] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:33.057286 12176 raft_consensus.cc:2454] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:33.057691 12216 leader_election.cc:302] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:33.057991 12454 raft_consensus.cc:2781] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:33.058068 12454 raft_consensus.cc:482] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:33.058115 12454 raft_consensus.cc:3037] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.060148 12454 raft_consensus.cc:504] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.060791 12454 leader_election.cc:288] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:33.061267 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbfbbbb346f413782d97580adfa82c1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:33.061514 12095 raft_consensus.cc:3037] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.061265 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbfbbbb346f413782d97580adfa82c1" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:33.061985 12176 raft_consensus.cc:3037] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.064493 12176 raft_consensus.cc:2454] T 4cbfbbbb346f413782d97580adfa82c1 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:33.064493 12095 raft_consensus.cc:2454] T 4cbfbbbb346f413782d97580adfa82c1 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:33.065042 12216 leader_election.cc:302] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:33.065033 12334 consensus_queue.cc:1035] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.065369 12454 raft_consensus.cc:2781] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:33.065506 12454 raft_consensus.cc:686] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:33.065797 12454 consensus_queue.cc:237] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.070683 12308 consensus_queue.cc:1035] T 9b8b568beaec444faa5e779e0b2059b1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.070897 12309 consensus_queue.cc:1035] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.071143 11993 catalog_manager.cc:5484] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:33.076118 12367 consensus_queue.cc:1035] T 969594f6d4b248758a81cc2b1b5bccca P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.105801 12308 raft_consensus.cc:482] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:33.106015 12308 raft_consensus.cc:504] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.106758 12308 leader_election.cc:288] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:33.107347 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b211abd4317f4be485f9db73fd55629e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f" is_pre_election: true
I20240502 21:29:33.107394 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b211abd4317f4be485f9db73fd55629e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:33.107582 12176 raft_consensus.cc:2454] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:33.107604 12095 raft_consensus.cc:2454] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 0.
I20240502 21:29:33.108049 12216 leader_election.cc:302] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:33.108444 12308 raft_consensus.cc:2781] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:33.108546 12308 raft_consensus.cc:482] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:33.108645 12308 raft_consensus.cc:3037] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.110697 12308 raft_consensus.cc:504] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.111343 12308 leader_election.cc:288] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:33.111852 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b211abd4317f4be485f9db73fd55629e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f04379252d4ecfab3a4916c61a984f"
I20240502 21:29:33.111894 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b211abd4317f4be485f9db73fd55629e" candidate_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:33.112075 12176 raft_consensus.cc:3037] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.112105 12095 raft_consensus.cc:3037] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.113919 12176 raft_consensus.cc:2454] T b211abd4317f4be485f9db73fd55629e P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:33.113919 12095 raft_consensus.cc:2454] T b211abd4317f4be485f9db73fd55629e P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7d3e3782d9416dae4e6479cb65c9a9 in term 1.
I20240502 21:29:33.114436 12219 leader_election.cc:302] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f04379252d4ecfab3a4916c61a984f, 2c7d3e3782d9416dae4e6479cb65c9a9; no voters:
I20240502 21:29:33.114709 12308 raft_consensus.cc:2781] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:33.114843 12308 raft_consensus.cc:686] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7d3e3782d9416dae4e6479cb65c9a9, State: Running, Role: LEADER
I20240502 21:29:33.115087 12308 consensus_queue.cc:237] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.118264 11994 catalog_manager.cc:5484] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67). New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } }
I20240502 21:29:33.139050 12301 raft_consensus.cc:482] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 21:29:33.139282 12301 raft_consensus.cc:504] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.140149 12301 leader_election.cc:288] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:33.140647 12095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df964090d307434981236aa967944cee" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0" is_pre_election: true
I20240502 21:29:33.140661 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df964090d307434981236aa967944cee" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" is_pre_election: true
I20240502 21:29:33.140872 12095 raft_consensus.cc:2454] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:33.140898 12251 raft_consensus.cc:2454] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 0.
I20240502 21:29:33.141253 12140 leader_election.cc:302] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:33.141506 12301 raft_consensus.cc:2781] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 21:29:33.141593 12301 raft_consensus.cc:482] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 21:29:33.141662 12301 raft_consensus.cc:3037] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.143934 12301 raft_consensus.cc:504] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.144661 12301 leader_election.cc:288] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Requested vote from peers 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241), 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:33.144951 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df964090d307434981236aa967944cee" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0672163bb07b41a1b14342a8efcb24a0"
I20240502 21:29:33.145154 12096 raft_consensus.cc:3037] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.146591 12251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df964090d307434981236aa967944cee" candidate_uuid: "17f04379252d4ecfab3a4916c61a984f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9"
I20240502 21:29:33.146785 12251 raft_consensus.cc:3037] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 21:29:33.148043 12096 raft_consensus.cc:2454] T df964090d307434981236aa967944cee P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:33.148479 12140 leader_election.cc:302] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0672163bb07b41a1b14342a8efcb24a0, 17f04379252d4ecfab3a4916c61a984f; no voters:
I20240502 21:29:33.148763 12301 raft_consensus.cc:2781] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 21:29:33.148878 12301 raft_consensus.cc:686] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Becoming Leader. State: Replica: 17f04379252d4ecfab3a4916c61a984f, State: Running, Role: LEADER
I20240502 21:29:33.149128 12301 consensus_queue.cc:237] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } }
I20240502 21:29:33.149536 12251 raft_consensus.cc:2454] T df964090d307434981236aa967944cee P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f04379252d4ecfab3a4916c61a984f in term 1.
I20240502 21:29:33.152626 11994 catalog_manager.cc:5484] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f04379252d4ecfab3a4916c61a984f (127.10.185.66). New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:33.156540 12301 consensus_queue.cc:1035] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.162684 12361 consensus_queue.cc:1035] T 04a7f75fa74945c09343353c01f4b31b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.204118 12366 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.210498 12316 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.211302 12361 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.217069 12301 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.220881 12316 consensus_queue.cc:1035] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.224725 12309 consensus_queue.cc:1035] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.249635 12334 consensus_queue.cc:1035] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.253895 12308 consensus_queue.cc:1035] T dd6c003ea2884ca9a1827814a4930742 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.263904 12309 consensus_queue.cc:1035] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.266527 12308 consensus_queue.cc:1035] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.272507 12367 consensus_queue.cc:1035] T a160b9e9063a4b95938cf7a5c944d77a P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.273032 12335 consensus_queue.cc:1035] T 1efbb27ede324d8d9005518cf8a66360 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.362860 12361 consensus_queue.cc:1035] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.367920 12301 consensus_queue.cc:1035] T 40d1079e722d4ca0b4db86807f9d7a7b P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.393626 12366 consensus_queue.cc:1035] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.397759 12367 consensus_queue.cc:1035] T 19293e7da2774ce4aa900e851931b6ae P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.450947 12452 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.455581 12334 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.473168 12334 consensus_queue.cc:1035] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.477002 12454 consensus_queue.cc:1035] T f5ef70a542484291bfb51c87b4d2f838 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.504068 12454 consensus_queue.cc:1035] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.507848 12454 consensus_queue.cc:1035] T b211abd4317f4be485f9db73fd55629e P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.641256 10981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 21:29:33.644770 12688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 21:29:33.645325 12689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:33.657534 12334 consensus_queue.cc:1035] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 21:29:33.657721 12691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 21:29:33.658918 10981 server_base.cc:1034] running on GCE node
I20240502 21:29:33.659225 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 21:29:33.659309 10981 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 21:29:33.659371 10981 hybrid_clock.cc:648] HybridClock initialized: now 1714685373659366 us; error 0 us; skew 500 ppm
I20240502 21:29:33.659638 10981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240502 21:29:33.662729 12335 consensus_queue.cc:1035] T 4cbfbbbb346f413782d97580adfa82c1 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.671309 10981 webserver.cc:460] Webserver started at http://127.10.185.68:43085/ using document root <none> and password file <none>
I20240502 21:29:33.671567 10981 fs_manager.cc:359] Metadata directory not provided
I20240502 21:29:33.671633 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 21:29:33.671747 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 21:29:33.672454 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-3-root/instance:
uuid: "20891959962b4799afe90a025ac94d3e"
format_stamp: "Formatted at 2024-05-02 21:29:33 on dist-test-slave-ccs3"
I20240502 21:29:33.675246 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20240502 21:29:33.677214 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:33.678011 10981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 21:29:33.678156 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-3-root
uuid: "20891959962b4799afe90a025ac94d3e"
format_stamp: "Formatted at 2024-05-02 21:29:33 on dist-test-slave-ccs3"
I20240502 21:29:33.678323 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 21:29:33.693243 10981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 21:29:33.693778 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 21:29:33.694339 10981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 21:29:33.695449 10981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 21:29:33.695514 10981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:33.695609 10981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 21:29:33.695649 10981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 21:29:33.716609 10981 rpc_server.cc:307] RPC server started. Bound to: 127.10.185.68:38619
I20240502 21:29:33.716609 12761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.185.68:38619 every 8 connection(s)
I20240502 21:29:33.718194 10981 auto_leader_rebalancer-test.cc:405] add a tserver: 20891959962b4799afe90a025ac94d3e
I20240502 21:29:33.725610 12762 heartbeater.cc:347] Connected to a master server at 127.10.185.126:42659
I20240502 21:29:33.725765 12762 heartbeater.cc:464] Registering TS with master...
I20240502 21:29:33.726207 12762 heartbeater.cc:511] Master 127.10.185.126:42659 requested a full tablet report, sending...
I20240502 21:29:33.727097 11994 ts_manager.cc:194] Registered new tserver with Master: 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619)
I20240502 21:29:33.727952 11994 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42824
I20240502 21:29:33.737711 12301 consensus_queue.cc:1035] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:33.740357 12301 consensus_queue.cc:1035] T df964090d307434981236aa967944cee P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 21:29:34.268306 12175 consensus_queue.cc:237] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.271622 12096 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.272327 12557 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.278028 12253 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.279772 12557 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.287585 12448 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.288487 12096 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.294795 11994 catalog_manager.cc:5484] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:34.301379 12253 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
W20240502 21:29:34.306062 12143 consensus_peers.cc:485] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af9adc5b5ce04374bf15c9088b4397d8. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:34.307907 12251 consensus_queue.cc:237] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.310469 12096 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.311178 12768 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.314567 12334 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.320401 12175 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.321710 12251 consensus_queue.cc:237] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.321926 12096 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.325708 12334 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.327427 12175 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.328991 11996 catalog_manager.cc:5484] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:34.333532 12098 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.334048 12334 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.343454 12175 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.344110 12334 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 21:29:34.345366 12219 consensus_peers.cc:485] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77279927ddf545e9a3216606b9bab54d. This is attempt 1: this message will repeat every 5th retry.
W20240502 21:29:34.345569 12219 consensus_peers.cc:485] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b158108cfdab4223b8f9d0226578de60. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:34.345777 12768 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.346510 12098 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.353173 11993 catalog_manager.cc:5484] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.355480 12176 consensus_queue.cc:237] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
W20240502 21:29:34.357785 12143 consensus_peers.cc:485] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad85ac6e9a824608ad426e1453c70237. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:34.357888 12098 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.358367 12448 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.360980 12175 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.364676 12253 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:34.366506 12448 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:34.368698 12787 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.371599 12098 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.372586 12251 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:34.376720 11993 catalog_manager.cc:5484] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:34.729207 12762 heartbeater.cc:503] Master 127.10.185.126:42659 was elected leader, sending a full tablet report...
I20240502 21:29:34.767923 12789 ts_tablet_manager.cc:902] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:34.770020 12789 tablet_copy_client.cc:308] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.67:34725
I20240502 21:29:34.778599 12264 tablet_copy_service.cc:140] P 2c7d3e3782d9416dae4e6479cb65c9a9: Received BeginTabletCopySession request for tablet 77279927ddf545e9a3216606b9bab54d from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:52858)
I20240502 21:29:34.778919 12264 tablet_copy_service.cc:161] P 2c7d3e3782d9416dae4e6479cb65c9a9: Beginning new tablet copy session on tablet 77279927ddf545e9a3216606b9bab54d from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:52858: session id = 20891959962b4799afe90a025ac94d3e-77279927ddf545e9a3216606b9bab54d
I20240502 21:29:34.782722 12264 tablet_copy_source_session.cc:205] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:34.784565 12789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77279927ddf545e9a3216606b9bab54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:34.790225 12789 tablet_copy_client.cc:783] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:34.790652 12789 tablet_copy_client.cc:647] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:34.793229 12789 tablet_copy_client.cc:523] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:34.797082 12789 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:34.805480 12789 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:34.806201 12789 tablet_bootstrap.cc:492] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:34.806505 12789 ts_tablet_manager.cc:1367] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20240502 21:29:34.807435 12797 ts_tablet_manager.cc:902] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:34.807425 12789 raft_consensus.cc:348] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.807791 12789 raft_consensus.cc:729] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:34.808097 12789 consensus_queue.cc:260] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.809262 12789 ts_tablet_manager.cc:1398] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 21:29:34.809672 12797 tablet_copy_client.cc:308] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.66:34017
I20240502 21:29:34.811239 12264 tablet_copy_service.cc:342] P 2c7d3e3782d9416dae4e6479cb65c9a9: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-77279927ddf545e9a3216606b9bab54d received from {username='slave'} at 127.0.0.1:52858
I20240502 21:29:34.811400 12264 tablet_copy_service.cc:433] P 2c7d3e3782d9416dae4e6479cb65c9a9: ending tablet copy session 20891959962b4799afe90a025ac94d3e-77279927ddf545e9a3216606b9bab54d on tablet 77279927ddf545e9a3216606b9bab54d with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:34.827378 12189 tablet_copy_service.cc:140] P 17f04379252d4ecfab3a4916c61a984f: Received BeginTabletCopySession request for tablet ad85ac6e9a824608ad426e1453c70237 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:55190)
I20240502 21:29:34.827592 12189 tablet_copy_service.cc:161] P 17f04379252d4ecfab3a4916c61a984f: Beginning new tablet copy session on tablet ad85ac6e9a824608ad426e1453c70237 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:55190: session id = 20891959962b4799afe90a025ac94d3e-ad85ac6e9a824608ad426e1453c70237
I20240502 21:29:34.829982 12189 tablet_copy_source_session.cc:205] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:34.831321 12797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad85ac6e9a824608ad426e1453c70237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:34.837337 12797 tablet_copy_client.cc:783] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:34.837594 12797 tablet_copy_client.cc:647] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:34.839581 12797 tablet_copy_client.cc:523] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:34.842330 12797 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:34.848464 12797 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:34.848840 12797 tablet_bootstrap.cc:492] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:34.849097 12797 ts_tablet_manager.cc:1367] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.004s
I20240502 21:29:34.849856 12797 raft_consensus.cc:348] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.850239 12797 raft_consensus.cc:729] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:34.850512 12797 consensus_queue.cc:260] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.851649 12797 ts_tablet_manager.cc:1398] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:34.853623 12189 tablet_copy_service.cc:342] P 17f04379252d4ecfab3a4916c61a984f: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-ad85ac6e9a824608ad426e1453c70237 received from {username='slave'} at 127.0.0.1:55190
I20240502 21:29:34.853739 12189 tablet_copy_service.cc:433] P 17f04379252d4ecfab3a4916c61a984f: ending tablet copy session 20891959962b4799afe90a025ac94d3e-ad85ac6e9a824608ad426e1453c70237 on tablet ad85ac6e9a824608ad426e1453c70237 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:34.880846 12797 ts_tablet_manager.cc:902] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:34.882423 12797 tablet_copy_client.cc:308] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.66:34017
I20240502 21:29:34.883206 12189 tablet_copy_service.cc:140] P 17f04379252d4ecfab3a4916c61a984f: Received BeginTabletCopySession request for tablet af9adc5b5ce04374bf15c9088b4397d8 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:55190)
I20240502 21:29:34.883373 12189 tablet_copy_service.cc:161] P 17f04379252d4ecfab3a4916c61a984f: Beginning new tablet copy session on tablet af9adc5b5ce04374bf15c9088b4397d8 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:55190: session id = 20891959962b4799afe90a025ac94d3e-af9adc5b5ce04374bf15c9088b4397d8
I20240502 21:29:34.885960 12189 tablet_copy_source_session.cc:205] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:34.887111 12797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9adc5b5ce04374bf15c9088b4397d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:34.894300 12797 tablet_copy_client.cc:783] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:34.894656 12797 tablet_copy_client.cc:647] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:34.897125 12797 tablet_copy_client.cc:523] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:34.909461 12797 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:34.916728 12797 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:34.917212 12797 tablet_bootstrap.cc:492] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:34.917469 12797 ts_tablet_manager.cc:1367] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20240502 21:29:34.918112 12797 raft_consensus.cc:348] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.918360 12797 raft_consensus.cc:729] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:34.918541 12797 consensus_queue.cc:260] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.919137 12797 ts_tablet_manager.cc:1398] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:34.919773 12189 tablet_copy_service.cc:342] P 17f04379252d4ecfab3a4916c61a984f: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-af9adc5b5ce04374bf15c9088b4397d8 received from {username='slave'} at 127.0.0.1:55190
I20240502 21:29:34.919889 12189 tablet_copy_service.cc:433] P 17f04379252d4ecfab3a4916c61a984f: ending tablet copy session 20891959962b4799afe90a025ac94d3e-af9adc5b5ce04374bf15c9088b4397d8 on tablet af9adc5b5ce04374bf15c9088b4397d8 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:34.929073 12797 ts_tablet_manager.cc:902] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:34.930653 12797 tablet_copy_client.cc:308] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.67:34725
I20240502 21:29:34.931368 12264 tablet_copy_service.cc:140] P 2c7d3e3782d9416dae4e6479cb65c9a9: Received BeginTabletCopySession request for tablet b158108cfdab4223b8f9d0226578de60 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:52858)
I20240502 21:29:34.931514 12264 tablet_copy_service.cc:161] P 2c7d3e3782d9416dae4e6479cb65c9a9: Beginning new tablet copy session on tablet b158108cfdab4223b8f9d0226578de60 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:52858: session id = 20891959962b4799afe90a025ac94d3e-b158108cfdab4223b8f9d0226578de60
I20240502 21:29:34.933993 12264 tablet_copy_source_session.cc:205] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:34.935206 12797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b158108cfdab4223b8f9d0226578de60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:34.950906 12797 tablet_copy_client.cc:783] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:34.951154 12797 tablet_copy_client.cc:647] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:34.952831 12797 tablet_copy_client.cc:523] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:34.955626 12797 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:34.961109 12797 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:34.961472 12797 tablet_bootstrap.cc:492] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:34.961737 12797 ts_tablet_manager.cc:1367] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.005s
I20240502 21:29:34.962472 12797 raft_consensus.cc:348] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.962723 12797 raft_consensus.cc:729] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:34.962911 12797 consensus_queue.cc:260] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:34.963426 12797 ts_tablet_manager.cc:1398] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:34.965317 12264 tablet_copy_service.cc:342] P 2c7d3e3782d9416dae4e6479cb65c9a9: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-b158108cfdab4223b8f9d0226578de60 received from {username='slave'} at 127.0.0.1:52858
I20240502 21:29:34.965435 12264 tablet_copy_service.cc:433] P 2c7d3e3782d9416dae4e6479cb65c9a9: ending tablet copy session 20891959962b4799afe90a025ac94d3e-b158108cfdab4223b8f9d0226578de60 on tablet b158108cfdab4223b8f9d0226578de60 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:35.301987 12736 raft_consensus.cc:1203] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:35.429658 12736 raft_consensus.cc:1203] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:35.435037 12736 raft_consensus.cc:1203] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:35.506620 12736 raft_consensus.cc:1203] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:35.836114 12787 raft_consensus.cc:1050] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:35.837323 12787 consensus_queue.cc:237] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:35.840464 12736 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:35.841171 12098 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:35.841171 12251 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:35.841593 12557 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:35.842257 12557 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:35.846349 12818 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:35.852684 12818 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.853330 12736 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.855999 12095 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.858242 11994 catalog_manager.cc:5484] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.859871 12251 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.870096 12175 consensus_queue.cc:237] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:35.872372 12095 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:35.872941 12736 raft_consensus.cc:1261] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:35.872954 12557 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:35.874935 12557 consensus_queue.cc:1035] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:35.875512 12787 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.877976 12736 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.879096 11983 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet af9adc5b5ce04374bf15c9088b4397d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:35.879812 12095 raft_consensus.cc:2932] T af9adc5b5ce04374bf15c9088b4397d8 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.885370 11995 catalog_manager.cc:5484] T af9adc5b5ce04374bf15c9088b4397d8 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:35.891796 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet af9adc5b5ce04374bf15c9088b4397d8 with delete_type TABLET_DATA_TOMBSTONED (TS 2c7d3e3782d9416dae4e6479cb65c9a9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52824
I20240502 21:29:35.893144 12828 tablet_replica.cc:330] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: stopping tablet replica
I20240502 21:29:35.893384 12828 raft_consensus.cc:2229] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:35.893669 12828 raft_consensus.cc:2258] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:35.894726 12828 ts_tablet_manager.cc:1875] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:35.903061 12828 ts_tablet_manager.cc:1888] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:35.903220 12828 log.cc:1196] T af9adc5b5ce04374bf15c9088b4397d8 P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root/wals/af9adc5b5ce04374bf15c9088b4397d8
I20240502 21:29:35.904084 11982 catalog_manager.cc:4830] TS 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): tablet af9adc5b5ce04374bf15c9088b4397d8 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:35.918990 12811 raft_consensus.cc:1050] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:35.919919 12811 consensus_queue.cc:237] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:35.922273 12736 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:35.922493 12095 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:35.922866 12768 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:35.923123 12175 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:35.923154 12334 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:35.923599 12811 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:35.927870 12768 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.928539 12175 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.932265 12096 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.937182 11996 catalog_manager.cc:5484] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.944763 12736 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.957532 12251 consensus_queue.cc:237] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:35.960714 12736 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:35.961210 12833 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:35.967097 12174 raft_consensus.cc:1261] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:35.969282 12835 consensus_queue.cc:1035] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:35.971760 12833 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.972437 12174 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.976038 11982 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 77279927ddf545e9a3216606b9bab54d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:35.977869 12736 raft_consensus.cc:2932] T 77279927ddf545e9a3216606b9bab54d P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.978662 11996 catalog_manager.cc:5484] T 77279927ddf545e9a3216606b9bab54d P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:35.982582 12087 tablet_service.cc:1514] Processing DeleteTablet for tablet 77279927ddf545e9a3216606b9bab54d with delete_type TABLET_DATA_TOMBSTONED (TS 0672163bb07b41a1b14342a8efcb24a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39156
I20240502 21:29:35.983747 12836 tablet_replica.cc:330] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: stopping tablet replica
I20240502 21:29:35.983955 12836 raft_consensus.cc:2229] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:35.984310 12836 raft_consensus.cc:2258] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:35.985373 12836 ts_tablet_manager.cc:1875] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:35.989858 12836 ts_tablet_manager.cc:1888] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:35.989977 12836 log.cc:1196] T 77279927ddf545e9a3216606b9bab54d P 0672163bb07b41a1b14342a8efcb24a0: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root/wals/77279927ddf545e9a3216606b9bab54d
I20240502 21:29:35.990702 11980 catalog_manager.cc:4830] TS 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): tablet 77279927ddf545e9a3216606b9bab54d (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:36.009573 12787 raft_consensus.cc:1050] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:36.010345 12787 consensus_queue.cc:237] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:36.012367 12252 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:36.012380 12736 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:36.012873 12820 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:36.013132 12818 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:36.014003 12834 raft_consensus.cc:1050] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:36.014791 12834 consensus_queue.cc:237] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:36.018368 12736 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:36.018424 12095 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:36.018683 12096 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:36.019111 12811 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:36.019243 12174 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:36.019925 12833 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:36.020124 12820 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.020484 12833 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:36.021071 12787 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:36.023581 12095 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.024834 12251 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.026470 12811 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.027822 12176 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.032243 12736 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.032657 11993 catalog_manager.cc:5484] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.032812 12095 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.039559 11995 catalog_manager.cc:5484] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:36.041141 12736 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.042383 12176 consensus_queue.cc:237] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:36.047123 12251 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:36.047683 12818 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:36.046893 12736 raft_consensus.cc:1261] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:36.049254 12818 consensus_queue.cc:1035] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:36.050527 12557 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.051407 12253 consensus_queue.cc:237] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:36.053470 12736 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.054248 11983 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ad85ac6e9a824608ad426e1453c70237 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:36.055627 12253 raft_consensus.cc:2932] T ad85ac6e9a824608ad426e1453c70237 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.056214 12737 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:36.058291 12096 raft_consensus.cc:1261] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:36.058437 12768 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:36.058967 12768 consensus_queue.cc:1035] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:36.058048 11996 catalog_manager.cc:5484] T ad85ac6e9a824608ad426e1453c70237 P 20891959962b4799afe90a025ac94d3e reported cstate change: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.061463 12833 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.062855 12096 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.063006 12736 raft_consensus.cc:2932] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.064987 11982 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b158108cfdab4223b8f9d0226578de60 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:36.066378 12087 tablet_service.cc:1514] Processing DeleteTablet for tablet ad85ac6e9a824608ad426e1453c70237 with delete_type TABLET_DATA_TOMBSTONED (TS 0672163bb07b41a1b14342a8efcb24a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39156
I20240502 21:29:36.066699 12836 tablet_replica.cc:330] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: stopping tablet replica
I20240502 21:29:36.066915 12836 raft_consensus.cc:2229] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:36.067111 12836 raft_consensus.cc:2258] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:36.067946 12836 ts_tablet_manager.cc:1875] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:36.071215 11996 catalog_manager.cc:5484] T b158108cfdab4223b8f9d0226578de60 P 20891959962b4799afe90a025ac94d3e reported cstate change: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:36.073701 12836 ts_tablet_manager.cc:1888] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:36.073793 12836 log.cc:1196] T ad85ac6e9a824608ad426e1453c70237 P 0672163bb07b41a1b14342a8efcb24a0: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root/wals/ad85ac6e9a824608ad426e1453c70237
I20240502 21:29:36.074327 11980 catalog_manager.cc:4830] TS 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): tablet ad85ac6e9a824608ad426e1453c70237 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:36.076602 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet b158108cfdab4223b8f9d0226578de60 with delete_type TABLET_DATA_TOMBSTONED (TS 17f04379252d4ecfab3a4916c61a984f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55152
I20240502 21:29:36.078724 12845 tablet_replica.cc:330] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: stopping tablet replica
I20240502 21:29:36.078963 12845 raft_consensus.cc:2229] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:36.079226 12845 raft_consensus.cc:2258] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:36.081595 12845 ts_tablet_manager.cc:1875] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:36.086043 12845 ts_tablet_manager.cc:1888] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:36.086141 12845 log.cc:1196] T b158108cfdab4223b8f9d0226578de60 P 17f04379252d4ecfab3a4916c61a984f: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root/wals/b158108cfdab4223b8f9d0226578de60
I20240502 21:29:36.086771 11983 catalog_manager.cc:4830] TS 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): tablet b158108cfdab4223b8f9d0226578de60 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:37.102296 12095 consensus_queue.cc:237] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.104676 12251 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.105085 12175 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.105415 12794 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:37.105706 12846 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:37.109081 12846 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.110605 12175 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
W20240502 21:29:37.113116 12066 consensus_peers.cc:485] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a729e66312d4e7fb61c9f7ecfe8d271. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:37.114073 12253 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.117420 11996 catalog_manager.cc:5484] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.120894 12095 consensus_queue.cc:237] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.124922 12253 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.125523 12175 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.125510 12847 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 21:29:37.125871 12066 consensus_peers.cc:485] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a1a147afb591461387bfe6ed7587a6ba. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:37.126168 12847 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:37.128607 12860 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.129623 12253 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.129768 12175 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.134913 12174 consensus_queue.cc:237] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.134992 11993 catalog_manager.cc:5484] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:37.149430 12096 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.149848 12253 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 21:29:37.149504 12143 consensus_peers.cc:485] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa48c97f4426444585651368f28b265a. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:37.150050 12864 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:37.150381 12820 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:37.155885 12864 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.156610 12096 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.161309 12174 consensus_queue.cc:237] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.161628 12253 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.162859 11996 catalog_manager.cc:5484] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:37.164116 12251 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.164597 12818 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:37.164777 12095 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:37.165565 12820 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 21:29:37.166841 12143 consensus_peers.cc:485] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b8307fd623a499fa05cd9eca238ddc2. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:37.179369 12818 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.182180 12251 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.182147 12095 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:37.185840 11996 catalog_manager.cc:5484] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:37.558682 12871 ts_tablet_manager.cc:902] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241)
I20240502 21:29:37.560240 12871 tablet_copy_client.cc:308] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.65:44241
I20240502 21:29:37.567538 12111 tablet_copy_service.cc:140] P 0672163bb07b41a1b14342a8efcb24a0: Received BeginTabletCopySession request for tablet a1a147afb591461387bfe6ed7587a6ba from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:36250)
I20240502 21:29:37.567731 12111 tablet_copy_service.cc:161] P 0672163bb07b41a1b14342a8efcb24a0: Beginning new tablet copy session on tablet a1a147afb591461387bfe6ed7587a6ba from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:36250: session id = 20891959962b4799afe90a025ac94d3e-a1a147afb591461387bfe6ed7587a6ba
I20240502 21:29:37.570788 12111 tablet_copy_source_session.cc:205] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:37.572141 12871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a147afb591461387bfe6ed7587a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:37.577324 12871 tablet_copy_client.cc:783] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:37.577587 12871 tablet_copy_client.cc:647] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:37.580317 12871 tablet_copy_client.cc:523] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:37.584486 12878 ts_tablet_manager.cc:902] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:37.585152 12871 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:37.592634 12878 tablet_copy_client.cc:308] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.66:34017
I20240502 21:29:37.593483 12189 tablet_copy_service.cc:140] P 17f04379252d4ecfab3a4916c61a984f: Received BeginTabletCopySession request for tablet aa48c97f4426444585651368f28b265a from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:55190)
I20240502 21:29:37.593638 12189 tablet_copy_service.cc:161] P 17f04379252d4ecfab3a4916c61a984f: Beginning new tablet copy session on tablet aa48c97f4426444585651368f28b265a from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:55190: session id = 20891959962b4799afe90a025ac94d3e-aa48c97f4426444585651368f28b265a
I20240502 21:29:37.593812 12871 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:37.594198 12871 tablet_bootstrap.cc:492] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:37.594393 12871 ts_tablet_manager.cc:1367] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20240502 21:29:37.595216 12871 raft_consensus.cc:348] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.595465 12871 raft_consensus.cc:729] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:37.595654 12871 consensus_queue.cc:260] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.596851 12189 tablet_copy_source_session.cc:205] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:37.597867 12871 ts_tablet_manager.cc:1398] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240502 21:29:37.598201 12878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa48c97f4426444585651368f28b265a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:37.598588 12111 tablet_copy_service.cc:342] P 0672163bb07b41a1b14342a8efcb24a0: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-a1a147afb591461387bfe6ed7587a6ba received from {username='slave'} at 127.0.0.1:36250
I20240502 21:29:37.598685 12111 tablet_copy_service.cc:433] P 0672163bb07b41a1b14342a8efcb24a0: ending tablet copy session 20891959962b4799afe90a025ac94d3e-a1a147afb591461387bfe6ed7587a6ba on tablet a1a147afb591461387bfe6ed7587a6ba with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:37.605720 12878 tablet_copy_client.cc:783] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:37.605954 12878 tablet_copy_client.cc:647] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:37.608076 12878 tablet_copy_client.cc:523] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:37.611537 12878 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:37.617306 12878 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:37.617730 12878 tablet_bootstrap.cc:492] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:37.617969 12878 ts_tablet_manager.cc:1367] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.001s
I20240502 21:29:37.618785 12878 raft_consensus.cc:348] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.619009 12878 raft_consensus.cc:729] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:37.619189 12878 consensus_queue.cc:260] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.620008 12878 ts_tablet_manager.cc:1398] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240502 21:29:37.620703 12189 tablet_copy_service.cc:342] P 17f04379252d4ecfab3a4916c61a984f: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-aa48c97f4426444585651368f28b265a received from {username='slave'} at 127.0.0.1:55190
I20240502 21:29:37.620793 12189 tablet_copy_service.cc:433] P 17f04379252d4ecfab3a4916c61a984f: ending tablet copy session 20891959962b4799afe90a025ac94d3e-aa48c97f4426444585651368f28b265a on tablet aa48c97f4426444585651368f28b265a with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:37.636245 12878 ts_tablet_manager.cc:902] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:37.639199 12878 tablet_copy_client.cc:308] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.66:34017
I20240502 21:29:37.651299 12189 tablet_copy_service.cc:140] P 17f04379252d4ecfab3a4916c61a984f: Received BeginTabletCopySession request for tablet 1b8307fd623a499fa05cd9eca238ddc2 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:55190)
I20240502 21:29:37.651505 12189 tablet_copy_service.cc:161] P 17f04379252d4ecfab3a4916c61a984f: Beginning new tablet copy session on tablet 1b8307fd623a499fa05cd9eca238ddc2 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:55190: session id = 20891959962b4799afe90a025ac94d3e-1b8307fd623a499fa05cd9eca238ddc2
I20240502 21:29:37.654069 12189 tablet_copy_source_session.cc:205] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:37.655660 12878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8307fd623a499fa05cd9eca238ddc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:37.660893 12878 tablet_copy_client.cc:783] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:37.661118 12878 tablet_copy_client.cc:647] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:37.663244 12878 tablet_copy_client.cc:523] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:37.666514 12878 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:37.672266 12878 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:37.672664 12878 tablet_bootstrap.cc:492] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:37.672941 12878 ts_tablet_manager.cc:1367] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.003s
I20240502 21:29:37.673602 12878 raft_consensus.cc:348] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.673866 12878 raft_consensus.cc:729] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:37.674065 12878 consensus_queue.cc:260] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.674616 12878 ts_tablet_manager.cc:1398] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:37.675485 12189 tablet_copy_service.cc:342] P 17f04379252d4ecfab3a4916c61a984f: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-1b8307fd623a499fa05cd9eca238ddc2 received from {username='slave'} at 127.0.0.1:55190
I20240502 21:29:37.675608 12189 tablet_copy_service.cc:433] P 17f04379252d4ecfab3a4916c61a984f: ending tablet copy session 20891959962b4799afe90a025ac94d3e-1b8307fd623a499fa05cd9eca238ddc2 on tablet 1b8307fd623a499fa05cd9eca238ddc2 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:37.717764 12878 ts_tablet_manager.cc:902] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241)
I20240502 21:29:37.719178 12878 tablet_copy_client.cc:308] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.65:44241
I20240502 21:29:37.719818 12111 tablet_copy_service.cc:140] P 0672163bb07b41a1b14342a8efcb24a0: Received BeginTabletCopySession request for tablet 4a729e66312d4e7fb61c9f7ecfe8d271 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:36250)
I20240502 21:29:37.720008 12111 tablet_copy_service.cc:161] P 0672163bb07b41a1b14342a8efcb24a0: Beginning new tablet copy session on tablet 4a729e66312d4e7fb61c9f7ecfe8d271 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:36250: session id = 20891959962b4799afe90a025ac94d3e-4a729e66312d4e7fb61c9f7ecfe8d271
I20240502 21:29:37.722652 12111 tablet_copy_source_session.cc:205] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:37.723717 12878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a729e66312d4e7fb61c9f7ecfe8d271. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:37.728518 12878 tablet_copy_client.cc:783] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:37.728767 12878 tablet_copy_client.cc:647] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:37.730253 12878 tablet_copy_client.cc:523] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:37.733126 12878 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:37.738840 12878 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:37.739270 12878 tablet_bootstrap.cc:492] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:37.739554 12878 ts_tablet_manager.cc:1367] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240502 21:29:37.740281 12878 raft_consensus.cc:348] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.740528 12878 raft_consensus.cc:729] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:37.740712 12878 consensus_queue.cc:260] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:37.741176 12878 ts_tablet_manager.cc:1398] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:37.741931 12111 tablet_copy_service.cc:342] P 0672163bb07b41a1b14342a8efcb24a0: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-4a729e66312d4e7fb61c9f7ecfe8d271 received from {username='slave'} at 127.0.0.1:36250
I20240502 21:29:37.742033 12111 tablet_copy_service.cc:433] P 0672163bb07b41a1b14342a8efcb24a0: ending tablet copy session 20891959962b4799afe90a025ac94d3e-4a729e66312d4e7fb61c9f7ecfe8d271 on tablet 4a729e66312d4e7fb61c9f7ecfe8d271 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:38.000228 12736 raft_consensus.cc:1203] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:38.170101 12736 raft_consensus.cc:1203] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:38.185983 12736 raft_consensus.cc:1203] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:38.247251 12736 raft_consensus.cc:1203] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:38.553748 12897 raft_consensus.cc:1050] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:38.554627 12897 consensus_queue.cc:237] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.557056 12736 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.557371 12096 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.557395 12253 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.558108 12899 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.558436 12867 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.558698 12864 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.570215 12897 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.571112 12736 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.576658 11995 catalog_manager.cc:5484] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.576831 12096 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.578402 12253 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.588593 12175 consensus_queue.cc:237] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.591249 12736 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.591454 12096 raft_consensus.cc:1261] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.592715 12899 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.593010 12897 consensus_queue.cc:1035] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.595883 12867 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.596454 12096 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.597069 12736 raft_consensus.cc:2932] T aa48c97f4426444585651368f28b265a P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.601787 11983 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet aa48c97f4426444585651368f28b265a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:38.603729 11993 catalog_manager.cc:5484] T aa48c97f4426444585651368f28b265a P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:38.609141 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet aa48c97f4426444585651368f28b265a with delete_type TABLET_DATA_TOMBSTONED (TS 2c7d3e3782d9416dae4e6479cb65c9a9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52824
I20240502 21:29:38.609947 12909 tablet_replica.cc:330] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: stopping tablet replica
I20240502 21:29:38.610165 12909 raft_consensus.cc:2229] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:38.610510 12909 raft_consensus.cc:2258] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:38.611924 12909 ts_tablet_manager.cc:1875] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:38.612723 12897 raft_consensus.cc:1050] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:38.613566 12897 consensus_queue.cc:237] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.616061 12736 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.616573 12251 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.616662 12899 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.617159 12867 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.618754 12909 ts_tablet_manager.cc:1888] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:38.618883 12909 log.cc:1196] T aa48c97f4426444585651368f28b265a P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root/wals/aa48c97f4426444585651368f28b265a
I20240502 21:29:38.619269 12096 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.619706 11982 catalog_manager.cc:4830] TS 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): tablet aa48c97f4426444585651368f28b265a (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:38.619721 12867 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.621507 12867 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.622968 12251 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.626956 12736 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.629652 11996 catalog_manager.cc:5484] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:38.634193 12096 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.639086 12176 consensus_queue.cc:237] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.640910 12251 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.641458 12899 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.642413 12736 raft_consensus.cc:1261] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.642894 12899 consensus_queue.cc:1035] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.643662 12897 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.647603 12251 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.648221 11983 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1b8307fd623a499fa05cd9eca238ddc2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:38.648878 12736 raft_consensus.cc:2932] T 1b8307fd623a499fa05cd9eca238ddc2 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.651274 12898 raft_consensus.cc:1050] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:38.651188 11994 catalog_manager.cc:5484] T 1b8307fd623a499fa05cd9eca238ddc2 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:38.654358 12898 consensus_queue.cc:237] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.655795 12087 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b8307fd623a499fa05cd9eca238ddc2 with delete_type TABLET_DATA_TOMBSTONED (TS 0672163bb07b41a1b14342a8efcb24a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39156
I20240502 21:29:38.658730 12915 tablet_replica.cc:330] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: stopping tablet replica
I20240502 21:29:38.658967 12915 raft_consensus.cc:2229] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:38.659194 12915 raft_consensus.cc:2258] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:38.660456 12915 ts_tablet_manager.cc:1875] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:38.660529 12176 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.660714 12253 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.660986 12736 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.661252 12898 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.661595 12847 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.662164 12847 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.673121 12915 ts_tablet_manager.cc:1888] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:38.674563 12915 log.cc:1196] T 1b8307fd623a499fa05cd9eca238ddc2 P 0672163bb07b41a1b14342a8efcb24a0: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root/wals/1b8307fd623a499fa05cd9eca238ddc2
I20240502 21:29:38.675480 11980 catalog_manager.cc:4830] TS 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): tablet 1b8307fd623a499fa05cd9eca238ddc2 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:38.676787 12847 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.681373 12251 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.681622 12176 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.682511 11994 catalog_manager.cc:5484] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:38.682721 12736 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.691668 12096 consensus_queue.cc:237] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.693802 12736 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.693958 12251 raft_consensus.cc:1261] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.694298 12847 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.694787 12847 consensus_queue.cc:1035] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.696820 12846 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.699441 11980 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a1a147afb591461387bfe6ed7587a6ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:38.701264 12736 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.702378 11994 catalog_manager.cc:5484] T a1a147afb591461387bfe6ed7587a6ba P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:38.701282 12251 raft_consensus.cc:2932] T a1a147afb591461387bfe6ed7587a6ba P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.706668 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a147afb591461387bfe6ed7587a6ba with delete_type TABLET_DATA_TOMBSTONED (TS 17f04379252d4ecfab3a4916c61a984f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55152
I20240502 21:29:38.707484 12919 tablet_replica.cc:330] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: stopping tablet replica
I20240502 21:29:38.707701 12919 raft_consensus.cc:2229] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:38.708472 12919 raft_consensus.cc:2258] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:38.710413 12919 ts_tablet_manager.cc:1875] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:38.717113 12919 ts_tablet_manager.cc:1888] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:38.717265 12919 log.cc:1196] T a1a147afb591461387bfe6ed7587a6ba P 17f04379252d4ecfab3a4916c61a984f: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root/wals/a1a147afb591461387bfe6ed7587a6ba
I20240502 21:29:38.718118 11983 catalog_manager.cc:4830] TS 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): tablet a1a147afb591461387bfe6ed7587a6ba (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:38.862382 12898 raft_consensus.cc:1050] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:38.863268 12898 consensus_queue.cc:237] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.865641 12736 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.865736 12176 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.866343 12251 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:38.866408 12847 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.867053 12846 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.867657 12846 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:38.874054 12847 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.877666 12736 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.879820 12176 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.881585 11993 catalog_manager.cc:5484] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:38.882772 12251 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.887116 12096 consensus_queue.cc:237] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:38.889139 12736 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.890024 12175 raft_consensus.cc:1261] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:38.889616 12898 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20240502 21:29:38.892130 12846 consensus_queue.cc:1035] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:38.894742 12846 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.895373 12176 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.896037 12736 raft_consensus.cc:2932] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.898612 11980 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a729e66312d4e7fb61c9f7ecfe8d271 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:38.901854 11994 catalog_manager.cc:5484] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:38.905321 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a729e66312d4e7fb61c9f7ecfe8d271 with delete_type TABLET_DATA_TOMBSTONED (TS 2c7d3e3782d9416dae4e6479cb65c9a9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52824
I20240502 21:29:38.905707 12909 tablet_replica.cc:330] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: stopping tablet replica
I20240502 21:29:38.905925 12909 raft_consensus.cc:2229] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:38.906169 12909 raft_consensus.cc:2258] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:38.907025 12909 ts_tablet_manager.cc:1875] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:38.912020 12909 ts_tablet_manager.cc:1888] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:38.912128 12909 log.cc:1196] T 4a729e66312d4e7fb61c9f7ecfe8d271 P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root/wals/4a729e66312d4e7fb61c9f7ecfe8d271
I20240502 21:29:38.912707 11982 catalog_manager.cc:4830] TS 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): tablet 4a729e66312d4e7fb61c9f7ecfe8d271 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:39.923393 12252 consensus_queue.cc:237] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:39.925947 12175 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:39.925948 12098 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:39.926797 12928 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.927129 12334 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 21:29:39.927364 12219 consensus_peers.cc:485] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e288a51b369a41ffa0a8d2dedb849c24. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:39.930130 12928 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.930713 12098 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.934439 12175 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.938282 11993 catalog_manager.cc:5484] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.938561 12098 consensus_queue.cc:237] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:39.942447 12252 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 21:29:39.942492 12066 consensus_peers.cc:485] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9758cbdd5b845d6883c6e34d0e3f6a6. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:39.942988 12847 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.944206 12175 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:39.945027 12847 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.948369 12847 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.952183 12176 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.953812 12175 consensus_queue.cc:237] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:39.965209 12098 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:39.965044 12251 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.966490 12252 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 21:29:39.968590 12143 consensus_peers.cc:485] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 244c153efb7943f48ddee849bb0187c4. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:39.968919 12925 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.970976 12899 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.974123 12937 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.974869 12252 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.977706 12096 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.984685 11994 catalog_manager.cc:5484] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:39.985280 11994 catalog_manager.cc:5484] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.985453 12252 consensus_queue.cc:237] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:39.988662 12096 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 21:29:39.989096 12219 consensus_peers.cc:485] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 434ffa8e08cf45f6905eb6aaaab820ca. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:39.989137 12175 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:39.989657 12896 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.989980 12928 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:39.993824 12928 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.994380 12096 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:39.995776 12176 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:40.001408 11994 catalog_manager.cc:5484] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:40.360499 12946 ts_tablet_manager.cc:902] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:40.362429 12946 tablet_copy_client.cc:308] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.67:34725
I20240502 21:29:40.363080 12264 tablet_copy_service.cc:140] P 2c7d3e3782d9416dae4e6479cb65c9a9: Received BeginTabletCopySession request for tablet e288a51b369a41ffa0a8d2dedb849c24 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:52858)
I20240502 21:29:40.363245 12264 tablet_copy_service.cc:161] P 2c7d3e3782d9416dae4e6479cb65c9a9: Beginning new tablet copy session on tablet e288a51b369a41ffa0a8d2dedb849c24 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:52858: session id = 20891959962b4799afe90a025ac94d3e-e288a51b369a41ffa0a8d2dedb849c24
I20240502 21:29:40.366133 12264 tablet_copy_source_session.cc:205] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:40.367589 12946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e288a51b369a41ffa0a8d2dedb849c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:40.375427 12946 tablet_copy_client.cc:783] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:40.375689 12946 tablet_copy_client.cc:647] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:40.377198 12946 tablet_copy_client.cc:523] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:40.380167 12946 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:40.385643 12946 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:40.386107 12946 tablet_bootstrap.cc:492] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:40.386379 12946 ts_tablet_manager.cc:1367] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20240502 21:29:40.387298 12946 raft_consensus.cc:348] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.387588 12946 raft_consensus.cc:729] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:40.387777 12946 consensus_queue.cc:260] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.388914 12946 ts_tablet_manager.cc:1398] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 21:29:40.389946 12264 tablet_copy_service.cc:342] P 2c7d3e3782d9416dae4e6479cb65c9a9: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-e288a51b369a41ffa0a8d2dedb849c24 received from {username='slave'} at 127.0.0.1:52858
I20240502 21:29:40.390058 12264 tablet_copy_service.cc:433] P 2c7d3e3782d9416dae4e6479cb65c9a9: ending tablet copy session 20891959962b4799afe90a025ac94d3e-e288a51b369a41ffa0a8d2dedb849c24 on tablet e288a51b369a41ffa0a8d2dedb849c24 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:40.536648 12946 ts_tablet_manager.cc:902] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
I20240502 21:29:40.540709 12946 tablet_copy_client.cc:308] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.66:34017
I20240502 21:29:40.541448 12189 tablet_copy_service.cc:140] P 17f04379252d4ecfab3a4916c61a984f: Received BeginTabletCopySession request for tablet 244c153efb7943f48ddee849bb0187c4 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:55190)
I20240502 21:29:40.541604 12189 tablet_copy_service.cc:161] P 17f04379252d4ecfab3a4916c61a984f: Beginning new tablet copy session on tablet 244c153efb7943f48ddee849bb0187c4 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:55190: session id = 20891959962b4799afe90a025ac94d3e-244c153efb7943f48ddee849bb0187c4
I20240502 21:29:40.544206 12189 tablet_copy_source_session.cc:205] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:40.545249 12946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 244c153efb7943f48ddee849bb0187c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:40.551674 12946 tablet_copy_client.cc:783] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:40.551899 12946 tablet_copy_client.cc:647] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:40.553730 12946 tablet_copy_client.cc:523] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:40.556205 12957 ts_tablet_manager.cc:902] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241)
I20240502 21:29:40.556921 12946 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:40.558513 12957 tablet_copy_client.cc:308] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.65:44241
I20240502 21:29:40.559134 12111 tablet_copy_service.cc:140] P 0672163bb07b41a1b14342a8efcb24a0: Received BeginTabletCopySession request for tablet c9758cbdd5b845d6883c6e34d0e3f6a6 from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:36250)
I20240502 21:29:40.559278 12111 tablet_copy_service.cc:161] P 0672163bb07b41a1b14342a8efcb24a0: Beginning new tablet copy session on tablet c9758cbdd5b845d6883c6e34d0e3f6a6 from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:36250: session id = 20891959962b4799afe90a025ac94d3e-c9758cbdd5b845d6883c6e34d0e3f6a6
I20240502 21:29:40.561620 12111 tablet_copy_source_session.cc:205] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:40.562120 12946 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:40.562469 12946 tablet_bootstrap.cc:492] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:40.562736 12946 ts_tablet_manager.cc:1367] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 21:29:40.562813 12957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9758cbdd5b845d6883c6e34d0e3f6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:40.563491 12946 raft_consensus.cc:348] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.563704 12946 raft_consensus.cc:729] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:40.563867 12946 consensus_queue.cc:260] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.564381 12946 ts_tablet_manager.cc:1398] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 21:29:40.567636 12957 tablet_copy_client.cc:783] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:40.567800 12957 tablet_copy_client.cc:647] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:40.570132 12189 tablet_copy_service.cc:342] P 17f04379252d4ecfab3a4916c61a984f: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-244c153efb7943f48ddee849bb0187c4 received from {username='slave'} at 127.0.0.1:55190
I20240502 21:29:40.570255 12189 tablet_copy_service.cc:433] P 17f04379252d4ecfab3a4916c61a984f: ending tablet copy session 20891959962b4799afe90a025ac94d3e-244c153efb7943f48ddee849bb0187c4 on tablet 244c153efb7943f48ddee849bb0187c4 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:40.571929 12957 tablet_copy_client.cc:523] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:40.575016 12957 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:40.580612 12957 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:40.581030 12957 tablet_bootstrap.cc:492] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:40.581285 12957 ts_tablet_manager.cc:1367] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.003s
I20240502 21:29:40.582027 12957 raft_consensus.cc:348] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.582252 12957 raft_consensus.cc:729] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:40.582433 12957 consensus_queue.cc:260] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.583128 12957 ts_tablet_manager.cc:1398] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20240502 21:29:40.583822 12111 tablet_copy_service.cc:342] P 0672163bb07b41a1b14342a8efcb24a0: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-c9758cbdd5b845d6883c6e34d0e3f6a6 received from {username='slave'} at 127.0.0.1:36250
I20240502 21:29:40.583943 12111 tablet_copy_service.cc:433] P 0672163bb07b41a1b14342a8efcb24a0: ending tablet copy session 20891959962b4799afe90a025ac94d3e-c9758cbdd5b845d6883c6e34d0e3f6a6 on tablet c9758cbdd5b845d6883c6e34d0e3f6a6 with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:40.590551 12957 ts_tablet_manager.cc:902] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725)
I20240502 21:29:40.591784 12957 tablet_copy_client.cc:308] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.185.67:34725
I20240502 21:29:40.592485 12264 tablet_copy_service.cc:140] P 2c7d3e3782d9416dae4e6479cb65c9a9: Received BeginTabletCopySession request for tablet 434ffa8e08cf45f6905eb6aaaab820ca from peer 20891959962b4799afe90a025ac94d3e ({username='slave'} at 127.0.0.1:52858)
I20240502 21:29:40.592634 12264 tablet_copy_service.cc:161] P 2c7d3e3782d9416dae4e6479cb65c9a9: Beginning new tablet copy session on tablet 434ffa8e08cf45f6905eb6aaaab820ca from peer 20891959962b4799afe90a025ac94d3e at {username='slave'} at 127.0.0.1:52858: session id = 20891959962b4799afe90a025ac94d3e-434ffa8e08cf45f6905eb6aaaab820ca
I20240502 21:29:40.594653 12264 tablet_copy_source_session.cc:205] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 21:29:40.595883 12957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434ffa8e08cf45f6905eb6aaaab820ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 21:29:40.601714 12957 tablet_copy_client.cc:783] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 0 data blocks...
I20240502 21:29:40.601974 12957 tablet_copy_client.cc:647] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: tablet copy: Starting download of 1 WAL segments...
I20240502 21:29:40.603612 12957 tablet_copy_client.cc:523] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 21:29:40.607110 12957 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: Bootstrap starting.
I20240502 21:29:40.613039 12957 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 21:29:40.613489 12957 tablet_bootstrap.cc:492] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: Bootstrap complete.
I20240502 21:29:40.613780 12957 ts_tablet_manager.cc:1367] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: Time spent bootstrapping tablet: real 0.007s user 0.008s sys 0.000s
I20240502 21:29:40.614629 12957 raft_consensus.cc:348] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.614884 12957 raft_consensus.cc:729] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20891959962b4799afe90a025ac94d3e, State: Initialized, Role: LEARNER
I20240502 21:29:40.615113 12957 consensus_queue.cc:260] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:40.615769 12957 ts_tablet_manager.cc:1398] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 21:29:40.616799 12264 tablet_copy_service.cc:342] P 2c7d3e3782d9416dae4e6479cb65c9a9: Request end of tablet copy session 20891959962b4799afe90a025ac94d3e-434ffa8e08cf45f6905eb6aaaab820ca received from {username='slave'} at 127.0.0.1:52858
I20240502 21:29:40.616904 12264 tablet_copy_service.cc:433] P 2c7d3e3782d9416dae4e6479cb65c9a9: ending tablet copy session 20891959962b4799afe90a025ac94d3e-434ffa8e08cf45f6905eb6aaaab820ca on tablet 434ffa8e08cf45f6905eb6aaaab820ca with peer 20891959962b4799afe90a025ac94d3e
I20240502 21:29:40.913559 12736 raft_consensus.cc:1203] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:40.963824 12736 raft_consensus.cc:1203] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:40.976615 12736 raft_consensus.cc:1203] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:41.095836 12736 raft_consensus.cc:1203] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 21:29:41.448959 12933 raft_consensus.cc:1050] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:41.449812 12933 consensus_queue.cc:237] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.461560 12736 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.462277 12174 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.462217 12974 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.461560 12253 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.463646 12974 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240502 21:29:41.464327 12933 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.473238 12974 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.474263 12251 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.477602 12174 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.481232 11996 catalog_manager.cc:5484] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.483053 12736 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.490438 12928 raft_consensus.cc:1050] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:41.491551 12928 consensus_queue.cc:237] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.494535 12736 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.494606 12176 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.495079 12928 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.495422 12896 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.495105 12095 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.497195 12969 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.503414 12896 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.505703 12925 raft_consensus.cc:1050] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:41.506657 12925 consensus_queue.cc:237] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.508400 12174 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.510452 12096 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.511092 12095 consensus_queue.cc:237] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.513507 12736 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.517292 12737 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.518827 11993 catalog_manager.cc:5484] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.519270 12970 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.520262 12175 raft_consensus.cc:1261] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.522089 12933 consensus_queue.cc:1035] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.522679 12933 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.525929 12251 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.526528 11980 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c9758cbdd5b845d6883c6e34d0e3f6a6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:41.526592 12981 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.527943 12098 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.528812 12176 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.529027 12251 consensus_queue.cc:237] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.531497 12098 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.531867 12735 raft_consensus.cc:1261] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.533370 12736 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.533449 12969 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.534073 12969 consensus_queue.cc:1035] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.534125 12983 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.535261 12969 raft_consensus.cc:1050] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9: attempting to promote NON_VOTER 20891959962b4799afe90a025ac94d3e to VOTER
I20240502 21:29:41.535941 12981 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.536325 12735 raft_consensus.cc:2932] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.538586 12969 consensus_queue.cc:237] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.540853 12928 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.543448 12098 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.547523 11993 catalog_manager.cc:5484] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 3 to 4, VOTER 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67) evicted. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:41.553436 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet c9758cbdd5b845d6883c6e34d0e3f6a6 with delete_type TABLET_DATA_TOMBSTONED (TS 2c7d3e3782d9416dae4e6479cb65c9a9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52824
I20240502 21:29:41.564991 11994 catalog_manager.cc:5484] T e288a51b369a41ffa0a8d2dedb849c24 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.570567 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet e288a51b369a41ffa0a8d2dedb849c24 with delete_type TABLET_DATA_TOMBSTONED (TS 17f04379252d4ecfab3a4916c61a984f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55152
I20240502 21:29:41.593261 11982 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e288a51b369a41ffa0a8d2dedb849c24 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:41.609993 12999 tablet_replica.cc:330] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: stopping tablet replica
I20240502 21:29:41.610394 12999 raft_consensus.cc:2229] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:41.610769 12999 raft_consensus.cc:2258] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:41.612212 12999 ts_tablet_manager.cc:1875] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20240502 21:29:41.612592 12219 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.185.68:38619, user_credentials={real_user=slave}} blocked reactor thread for 65766.2us
I20240502 21:29:41.620002 12737 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 LEARNER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.621057 12175 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.634239 12999 ts_tablet_manager.cc:1888] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:41.634418 12999 log.cc:1196] T e288a51b369a41ffa0a8d2dedb849c24 P 17f04379252d4ecfab3a4916c61a984f: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root/wals/e288a51b369a41ffa0a8d2dedb849c24
I20240502 21:29:41.634907 12737 raft_consensus.cc:2932] T e288a51b369a41ffa0a8d2dedb849c24 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.635478 11983 catalog_manager.cc:4830] TS 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): tablet e288a51b369a41ffa0a8d2dedb849c24 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:41.646745 12969 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20240502 21:29:41.647472 12216 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.185.65:44241, user_credentials={real_user=slave}} blocked reactor thread for 85586.2us
I20240502 21:29:41.648411 12096 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 21:29:41.649565 12997 tablet_replica.cc:330] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: stopping tablet replica
I20240502 21:29:41.649737 12896 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.652715 12997 raft_consensus.cc:2229] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:41.653791 12998 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 21:29:41.657964 12990 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.659570 12997 raft_consensus.cc:2258] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:41.659148 12736 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.661101 12997 ts_tablet_manager.cc:1875] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:41.660228 12099 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.672307 12998 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.676592 11994 catalog_manager.cc:5484] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.677006 12099 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.680233 12253 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.681234 11995 catalog_manager.cc:5484] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 reported cstate change: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 21:29:41.683894 12736 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.687053 12252 consensus_queue.cc:237] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.688956 12099 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.689584 12896 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.693275 12995 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.693881 12096 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.696504 12737 raft_consensus.cc:1261] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 2c7d3e3782d9416dae4e6479cb65c9a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.696708 12174 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20891959962b4799afe90a025ac94d3e (127.10.185.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.697773 12896 consensus_queue.cc:1035] T 434ffa8e08cf45f6905eb6aaaab820ca P 2c7d3e3782d9416dae4e6479cb65c9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.699604 12736 raft_consensus.cc:2932] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.700919 11982 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 434ffa8e08cf45f6905eb6aaaab820ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:41.693579 12175 consensus_queue.cc:237] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } }
I20240502 21:29:41.702733 12737 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.703214 12981 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.705096 12997 ts_tablet_manager.cc:1888] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:41.706465 12997 log.cc:1196] T c9758cbdd5b845d6883c6e34d0e3f6a6 P 2c7d3e3782d9416dae4e6479cb65c9a9: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-2-root/wals/c9758cbdd5b845d6883c6e34d0e3f6a6
I20240502 21:29:41.707218 12253 raft_consensus.cc:1261] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 21:29:41.707357 12981 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.707688 12925 consensus_queue.cc:1035] T 244c153efb7943f48ddee849bb0187c4 P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 21:29:41.708212 11982 catalog_manager.cc:4830] TS 2c7d3e3782d9416dae4e6479cb65c9a9 (127.10.185.67:34725): tablet c9758cbdd5b845d6883c6e34d0e3f6a6 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:41.708721 12736 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.710562 12253 raft_consensus.cc:2932] T 244c153efb7943f48ddee849bb0187c4 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.711383 11983 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 244c153efb7943f48ddee849bb0187c4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 21:29:41.719619 11996 catalog_manager.cc:5484] T 434ffa8e08cf45f6905eb6aaaab820ca P 20891959962b4799afe90a025ac94d3e reported cstate change: config changed from index 3 to 4, VOTER 17f04379252d4ecfab3a4916c61a984f (127.10.185.66) evicted. New cstate: current_term: 1 leader_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.720485 11996 catalog_manager.cc:5484] T 244c153efb7943f48ddee849bb0187c4 P 20891959962b4799afe90a025ac94d3e reported cstate change: config changed from index 3 to 4, VOTER 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65) evicted. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: false } } }
I20240502 21:29:41.725399 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet 434ffa8e08cf45f6905eb6aaaab820ca with delete_type TABLET_DATA_TOMBSTONED (TS 17f04379252d4ecfab3a4916c61a984f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55152
I20240502 21:29:41.725581 12087 tablet_service.cc:1514] Processing DeleteTablet for tablet 244c153efb7943f48ddee849bb0187c4 with delete_type TABLET_DATA_TOMBSTONED (TS 0672163bb07b41a1b14342a8efcb24a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39156
I20240502 21:29:41.725914 12999 tablet_replica.cc:330] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: stopping tablet replica
I20240502 21:29:41.726157 12999 raft_consensus.cc:2229] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:41.726508 12999 raft_consensus.cc:2258] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:41.727645 12999 ts_tablet_manager.cc:1875] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:41.736665 13004 tablet_replica.cc:330] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: stopping tablet replica
I20240502 21:29:41.736914 13004 raft_consensus.cc:2229] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 21:29:41.737260 13004 raft_consensus.cc:2258] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 21:29:41.738204 12999 ts_tablet_manager.cc:1888] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:41.738327 12999 log.cc:1196] T 434ffa8e08cf45f6905eb6aaaab820ca P 17f04379252d4ecfab3a4916c61a984f: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-1-root/wals/434ffa8e08cf45f6905eb6aaaab820ca
I20240502 21:29:41.738430 13004 ts_tablet_manager.cc:1875] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 21:29:41.739005 11983 catalog_manager.cc:4830] TS 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017): tablet 434ffa8e08cf45f6905eb6aaaab820ca (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:41.743381 13004 ts_tablet_manager.cc:1888] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 21:29:41.743497 13004 log.cc:1196] T 244c153efb7943f48ddee849bb0187c4 P 0672163bb07b41a1b14342a8efcb24a0: Deleting WAL directory at /tmp/dist-test-taskalNgv8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714685314041113-10981-0/minicluster-data/ts-0-root/wals/244c153efb7943f48ddee849bb0187c4
I20240502 21:29:41.744212 11980 catalog_manager.cc:4830] TS 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241): tablet 244c153efb7943f48ddee849bb0187c4 (table test-workload [id=ac5c3611cbbc479f88474c64c2be8100]) successfully deleted
I20240502 21:29:42.736460 12096 consensus_queue.cc:237] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:42.739115 12253 raft_consensus.cc:1261] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:42.739375 12175 raft_consensus.cc:1261] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:42.739805 12970 consensus_queue.cc:1035] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 21:29:42.740115 12066 consensus_peers.cc:485] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b56cd8f7bdee468b846b4f51e86e1f27. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:42.740604 12970 consensus_queue.cc:1035] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.742972 12970 raft_consensus.cc:2932] T b56cd8f7bdee468b846b4f51e86e1f27 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.744347 12252 raft_consensus.cc:2932] T b56cd8f7bdee468b846b4f51e86e1f27 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.744719 12175 raft_consensus.cc:2932] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.750881 12096 consensus_queue.cc:237] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:42.751539 11994 catalog_manager.cc:5484] T b56cd8f7bdee468b846b4f51e86e1f27 P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.754068 12252 raft_consensus.cc:1261] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 21:29:42.754160 12066 consensus_peers.cc:485] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6d88ae6514545bca16f20beb52b8e7b. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:42.754660 12175 raft_consensus.cc:1261] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:42.754668 13006 consensus_queue.cc:1035] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.755263 13008 consensus_queue.cc:1035] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.758059 13009 raft_consensus.cc:2932] T b6d88ae6514545bca16f20beb52b8e7b P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.759058 12175 raft_consensus.cc:2932] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.762763 12252 raft_consensus.cc:2932] T b6d88ae6514545bca16f20beb52b8e7b P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.764596 12174 consensus_queue.cc:237] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:42.770844 11996 catalog_manager.cc:5484] T b6d88ae6514545bca16f20beb52b8e7b P 17f04379252d4ecfab3a4916c61a984f reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } attrs { replace: true } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
W20240502 21:29:42.772715 12143 consensus_peers.cc:485] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 060e5a3ff13f40629080ec7036eb39ef. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:42.773490 12099 raft_consensus.cc:1261] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:42.773998 12252 raft_consensus.cc:1261] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 17f04379252d4ecfab3a4916c61a984f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:42.774950 12925 consensus_queue.cc:1035] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.777484 12981 consensus_queue.cc:1035] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.779392 12925 raft_consensus.cc:2932] T 060e5a3ff13f40629080ec7036eb39ef P 17f04379252d4ecfab3a4916c61a984f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.783056 12099 raft_consensus.cc:2932] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.784355 12096 consensus_queue.cc:237] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } }
I20240502 21:29:42.787109 12252 raft_consensus.cc:1261] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 21:29:42.787475 12066 consensus_peers.cc:485] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 -> Peer 20891959962b4799afe90a025ac94d3e (127.10.185.68:38619): Couldn't send request to peer 20891959962b4799afe90a025ac94d3e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5cc5520fead4142819e3842fd573287. This is attempt 1: this message will repeat every 5th retry.
I20240502 21:29:42.787782 13009 consensus_queue.cc:1035] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.790544 11994 catalog_manager.cc:5484] T 060e5a3ff13f40629080ec7036eb39ef P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "17f04379252d4ecfab3a4916c61a984f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.791234 12970 raft_consensus.cc:2932] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.792039 12252 raft_consensus.cc:2932] T b5cc5520fead4142819e3842fd573287 P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.795257 12253 raft_consensus.cc:2932] T 060e5a3ff13f40629080ec7036eb39ef P 2c7d3e3782d9416dae4e6479cb65c9a9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } attrs { replace: true } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:42.798389 11993 catalog_manager.cc:5484] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New cstate: current_term: 1 leader_uuid: "0672163bb07b41a1b14342a8efcb24a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 21:29:42.800029 12174 raft_consensus.cc:1261] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Refusing update from remote peer 0672163bb07b41a1b14342a8efcb24a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 21:29:42.800694 12970 consensus_queue.cc:1035] T b5cc5520fead4142819e3842fd573287 P 0672163bb07b41a1b14342a8efcb24a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 21:29:42.808076 12174 raft_consensus.cc:2932] T b5cc5520fead4142819e3842fd573287 P 17f04379252d4ecfab3a4916c61a984f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20891959962b4799afe90a025ac94d3e (127.10.185.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0672163bb07b41a1b14342a8efcb24a0" member_type: VOTER last_known_addr { host: "127.10.185.65" port: 44241 } } peers { permanent_uuid: "17f04379252d4ecfab3a4916c61a984f" member_type: VOTER last_known_addr { host: "127.10.185.66" port: 34017 } } peers { permanent_uuid: "2c7d3e3782d9416dae4e6479cb65c9a9" member_type: VOTER last_known_addr { host: "127.10.185.67" port: 34725 } attrs { replace: true } } peers { permanent_uuid: "20891959962b4799afe90a025ac94d3e" member_type: NON_VOTER last_known_addr { host: "127.10.185.68" port: 38619 } attrs { promote: true } } }
I20240502 21:29:43.200621 13027 ts_tablet_manager.cc:902] T b6d88ae6514545bca16f20beb52b8e7b P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241)
F20240502 21:29:43.201161 13027 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (init.queue_time_us, init.queue_time_us) (init.run_cpu_time_us, init.run_cpu_time_us) (init.run_wall_time_us, init.run_wall_time_us) (leader-initialization.queue_time_us, leader-initialization.queue_time_us) (leader-initialization.run_cpu_time_us, leader-initialization.run_cpu_time_us) (leader-initialization.run_wall_time_us, leader-initialization.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-1b8307fd623a499fa05cd9eca238ddc2.queue_time_us, tablet-download-pool-1b8307fd623a499fa05cd9eca238ddc2.queue_time_us) (tablet-download-pool-1b8307fd623a499fa05cd9eca238ddc2.run_cpu_time_us, tablet-download-pool-1b8307fd623a499fa05cd9eca238ddc2.run_cpu_time_us) (tablet-download-pool-1b8307fd623a499fa05cd9eca238ddc2.run_wall_time_us, tablet-download-pool-1b8307fd623a499fa05cd9eca238ddc2.run_wall_time_us) (tablet-download-pool-244c153efb7943f48ddee849bb0187c4.queue_time_us, tablet-download-pool-244c153efb7943f48ddee849bb0187c4.queue_time_us) (tablet-download-pool-244c153efb7943f48ddee849bb0187c4.run_cpu_time_us, tablet-download-pool-244c153efb7943f48ddee849bb0187c4.run_cpu_time_us) (tablet-download-pool-244c153efb7943f48ddee849bb0187c4.run_wall_time_us, tablet-download-pool-244c153efb7943f48ddee849bb0187c4.run_wall_time_us) (tablet-download-pool-434ffa8e08cf45f6905eb6aaaab820ca.queue_time_us, tablet-download-pool-434ffa8e08cf45f6905eb6aaaab820ca.queue_time_us) (tablet-download-pool-434ffa8e08cf45f6905eb6aaaab820ca.run_cpu_time_us, tablet-download-pool-434ffa8e08cf45f6905eb6aaaab820ca.run_cpu_time_us) (tablet-download-pool-434ffa8e08cf45f6905eb6aaaab820ca.run_wall_time_us, tablet-download-pool-434ffa8e08cf45f6905eb6aaaab820ca.run_wall_time_us) (tablet-download-pool-4a729e66312d4e7fb61c9f7ecfe8d271.queue_time_us, tablet-download-pool-4a729e66312d4e7fb61c9f7ecfe8d271.queue_time_us) (tablet-download-pool-4a729e66312d4e7fb61c9f7ecfe8d271.run_cpu_time_us, tablet-download-pool-4a729e66312d4e7fb61c9f7ecfe8d271.run_cpu_time_us) (tablet-download-pool-4a729e66312d4e7fb61c9f7ecfe8d271.run_wall_time_us, tablet-download-pool-4a729e66312d4e7fb61c9f7ecfe8d271.run_wall_time_us) (tablet-download-pool-77279927ddf545e9a3216606b9bab54d.queue_time_us, tablet-download-pool-77279927ddf545e9a3216606b9bab54d.queue_time_us) (tablet-download-pool-77279927ddf545e9a3216606b9bab54d.run_cpu_time_us, tablet-download-pool-77279927ddf545e9a3216606b9bab54d.run_cpu_time_us) (tablet-download-pool-77279927ddf545e9a3216606b9bab54d.run_wall_time_us, tablet-download-pool-77279927ddf545e9a3216606b9bab54d.run_wall_time_us) (tablet-download-pool-a1a147afb591461387bfe6ed7587a6ba.queue_time_us, tablet-download-pool-a1a147afb591461387bfe6ed7587a6ba.queue_time_us) (tablet-download-pool-a1a147afb591461387bfe6ed7587a6ba.run_cpu_time_us, tablet-download-pool-a1a147afb591461387bfe6ed7587a6ba.run_cpu_time_us) (tablet-download-pool-a1a147afb591461387bfe6ed7587a6ba.run_wall_time_us, tablet-download-pool-a1a147afb591461387bfe6ed7587a6ba.run_wall_time_us) (tablet-download-pool-aa48c97f4426444585651368f28b265a.queue_time_us, tablet-download-pool-aa48c97f4426444585651368f28b265a.queue_time_us) (tablet-download-pool-aa48c97f4426444585651368f28b265a.run_cpu_time_us, tablet-download-pool-aa48c97f4426444585651368f28b265a.run_cpu_time_us) (tablet-download-pool-aa48c97f4426444585651368f28b265a.run_wall_time_us, tablet-download-pool-aa48c97f4426444585651368f28b265a.run_wall_time_us) (tablet-download-pool-ad85ac6e9a824608ad426e1453c70237.queue_time_us, tablet-download-pool-ad85ac6e9a824608ad426e1453c70237.queue_time_us) (tablet-download-pool-ad85ac6e9a824608ad426e1453c70237.run_cpu_time_us, tablet-download-pool-ad85ac6e9a824608ad426e1453c70237.run_cpu_time_us) (tablet-download-pool-ad85ac6e9a824608ad426e1453c70237.run_wall_time_us, tablet-download-pool-ad85ac6e9a824608ad426e1453c70237.run_wall_time_us) (tablet-download-pool-af9adc5b5ce04374bf15c9088b4397d8.queue_time_us, tablet-download-pool-af9adc5b5ce04374bf15c9088b4397d8.queue_time_us) (tablet-download-pool-af9adc5b5ce04374bf15c9088b4397d8.run_cpu_time_us, tablet-download-pool-af9adc5b5ce04374bf15c9088b4397d8.run_cpu_time_us) (tablet-download-pool-af9adc5b5ce04374bf15c9088b4397d8.run_wall_time_us, tablet-download-pool-af9adc5b5ce04374bf15c9088b4397d8.run_wall_time_us) (tablet-download-pool-b158108cfdab4223b8f9d0226578de60.queue_time_us, tablet-download-pool-b158108cfdab4223b8f9d0226578de60.queue_time_us) (tablet-download-pool-b158108cfdab4223b8f9d0226578de60.run_cpu_time_us, tablet-download-pool-b158108cfdab4223b8f9d0226578de60.run_cpu_time_us) (tablet-download-pool-b158108cfdab4223b8f9d0226578de60.run_wall_time_us, tablet-download-pool-b158108cfdab4223b8f9d0226578de60.run_wall_time_us) (tablet-download-pool-b6d88ae6514545bca16f20beb52b8e7b.queue_time_us, tablet-download-pool-b6d88ae6514545bca16f20beb52b8e7b.queue_time_us) (tablet-download-pool-c9758cbdd5b845d6883c6e34d0e3f6a6.queue_time_us, tablet-download-pool-c9758cbdd5b845d6883c6e34d0e3f6a6.queue_time_us) (tablet-download-pool-c9758cbdd5b845d6883c6e34d0e3f6a6.run_cpu_time_us, tablet-download-pool-c9758cbdd5b845d6883c6e34d0e3f6a6.run_cpu_time_us) (tablet-download-pool-c9758cbdd5b845d6883c6e34d0e3f6a6.run_wall_time_us, tablet-download-pool-c9758cbdd5b845d6883c6e34d0e3f6a6.run_wall_time_us) (tablet-download-pool-e288a51b369a41ffa0a8d2dedb849c24.queue_time_us, tablet-download-pool-e288a51b369a41ffa0a8d2dedb849c24.queue_time_us) (tablet-download-pool-e288a51b369a41ffa0a8d2dedb849c24.run_cpu_time_us, tablet-download-pool-e288a51b369a41ffa0a8d2dedb849c24.run_cpu_time_us) (tablet-download-pool-e288a51b369a41ffa0a8d2dedb849c24.run_wall_time_us, tablet-download-pool-e288a51b369a41ffa0a8d2dedb849c24.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
*** Aborted at 1714685383 (unix time) try "date -d @1714685383" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800002ae5) received by PID 10981 (TID 0x7f83f9758700) from PID 10981; stack trace: ***
@ 0x7f84a5d5e980 (unknown) at ??:0
@ 0x7f849b0abfb7 gsignal at ??:0
@ 0x7f849b0ad921 abort at ??:0
@ 0x7f849db60dcd google::LogMessage::Fail() at ??:0
@ 0x7f849db64b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f849db607cc google::LogMessage::Flush() at ??:0
@ 0x7f849db61f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f849f1dd742 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f849f1a9f8b kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f849f1a3cd6 kudu::ThreadPoolBuilder::Build() at ??:0
I20240502 21:29:43.254587 13033 ts_tablet_manager.cc:902] T 060e5a3ff13f40629080ec7036eb39ef P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 17f04379252d4ecfab3a4916c61a984f (127.10.185.66:34017)
@ 0x7f84ae66f8de kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f84ae687daf kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f84ae7b2480 kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
I20240502 21:29:43.276147 13038 ts_tablet_manager.cc:902] T b56cd8f7bdee468b846b4f51e86e1f27 P 20891959962b4799afe90a025ac94d3e: Initiating tablet copy from peer 0672163bb07b41a1b14342a8efcb24a0 (127.10.185.65:44241)
@ 0x7f84ae7eaa8d kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f84ae7cd981 kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
@ 0x7f84ae7cd70e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f84b366f98d std::function<>::operator()() at ??:0
@ 0x7f849f1adb75 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f849f1b119c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f849f1b1019 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f84b366f98d std::function<>::operator()() at ??:0
@ 0x7f849f18c01e kudu::Thread::SuperviseThread() at ??:0
@ 0x7f84a5d536db start_thread at ??:0
@ 0x7f849b18e71f clone at ??:0