[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240503 02:12:35.664660 2677 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.157.126:34485
I20240503 02:12:35.666344 2677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:12:35.667623 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:12:35.680122 2685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:12:35.680543 2683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:12:35.681303 2682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:35.681999 2677 server_base.cc:1034] running on GCE node
I20240503 02:12:35.682829 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:12:35.682991 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:12:35.683076 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702355683057 us; error 0 us; skew 500 ppm
I20240503 02:12:35.683393 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:12:35.690644 2677 webserver.cc:460] Webserver started at http://127.2.157.126:34187/ using document root <none> and password file <none>
I20240503 02:12:35.691407 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:12:35.691501 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:12:35.691818 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:12:35.695158 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/master-0-root/instance:
uuid: "69e51a9327804440a82be2eab3b07e93"
format_stamp: "Formatted at 2024-05-03 02:12:35 on dist-test-slave-tz5s"
I20240503 02:12:35.700928 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20240503 02:12:35.704236 2690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:35.705874 2677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240503 02:12:35.706068 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/master-0-root
uuid: "69e51a9327804440a82be2eab3b07e93"
format_stamp: "Formatted at 2024-05-03 02:12:35 on dist-test-slave-tz5s"
I20240503 02:12:35.706255 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:12:35.720782 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:12:35.721741 2677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:12:35.722054 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:12:35.742216 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.126:34485
I20240503 02:12:35.742218 2741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.126:34485 every 8 connection(s)
I20240503 02:12:35.746840 2742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:35.759706 2742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93: Bootstrap starting.
I20240503 02:12:35.763229 2742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:35.764333 2742 log.cc:824] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93: Log is configured to *not* fsync() on all Append() calls
I20240503 02:12:35.766947 2742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93: No bootstrap required, opened a new log
I20240503 02:12:35.773916 2742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e51a9327804440a82be2eab3b07e93" member_type: VOTER }
I20240503 02:12:35.774250 2742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:35.774365 2742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e51a9327804440a82be2eab3b07e93, State: Initialized, Role: FOLLOWER
I20240503 02:12:35.775132 2742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e51a9327804440a82be2eab3b07e93" member_type: VOTER }
I20240503 02:12:35.775346 2742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:12:35.775481 2742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:12:35.775642 2742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:35.778251 2742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e51a9327804440a82be2eab3b07e93" member_type: VOTER }
I20240503 02:12:35.778754 2742 leader_election.cc:302] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69e51a9327804440a82be2eab3b07e93; no voters:
I20240503 02:12:35.779367 2742 leader_election.cc:288] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:12:35.779495 2745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:35.780148 2745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 1 LEADER]: Becoming Leader. State: Replica: 69e51a9327804440a82be2eab3b07e93, State: Running, Role: LEADER
I20240503 02:12:35.780737 2745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e51a9327804440a82be2eab3b07e93" member_type: VOTER }
I20240503 02:12:35.781453 2742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:12:35.783674 2746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69e51a9327804440a82be2eab3b07e93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e51a9327804440a82be2eab3b07e93" member_type: VOTER } }
I20240503 02:12:35.784008 2746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [sys.catalog]: This master's current role is: LEADER
I20240503 02:12:35.784255 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69e51a9327804440a82be2eab3b07e93. Latest consensus state: current_term: 1 leader_uuid: "69e51a9327804440a82be2eab3b07e93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e51a9327804440a82be2eab3b07e93" member_type: VOTER } }
I20240503 02:12:35.784583 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [sys.catalog]: This master's current role is: LEADER
I20240503 02:12:35.786109 2753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:12:35.791612 2753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:12:35.792222 2677 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240503 02:12:35.800784 2753 catalog_manager.cc:1347] Generated new cluster ID: 8299595ced7843ecbc17a53a92d7e4ef
I20240503 02:12:35.800902 2753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:12:35.837862 2753 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:12:35.839655 2753 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:12:35.857358 2753 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93: Generated new TSK 0
I20240503 02:12:35.858269 2753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:12:35.922237 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:12:35.926916 2764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:12:35.927944 2765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:12:35.928409 2767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:35.929291 2677 server_base.cc:1034] running on GCE node
I20240503 02:12:35.929610 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:12:35.929700 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:12:35.929765 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702355929742 us; error 0 us; skew 500 ppm
I20240503 02:12:35.929962 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:12:35.931389 2677 webserver.cc:460] Webserver started at http://127.2.157.65:42435/ using document root <none> and password file <none>
I20240503 02:12:35.931643 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:12:35.931726 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:12:35.931882 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:12:35.932574 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-0-root/instance:
uuid: "0906dca094bd43a190401b5abde7bd21"
format_stamp: "Formatted at 2024-05-03 02:12:35 on dist-test-slave-tz5s"
I20240503 02:12:35.939358 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20240503 02:12:35.941244 2772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:35.941697 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20240503 02:12:35.941859 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-0-root
uuid: "0906dca094bd43a190401b5abde7bd21"
format_stamp: "Formatted at 2024-05-03 02:12:35 on dist-test-slave-tz5s"
I20240503 02:12:35.942008 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:12:35.961059 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:12:35.961902 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:12:35.962945 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:12:35.964802 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:12:35.964964 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:35.965137 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:12:35.965250 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:35.991041 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.65:39937
I20240503 02:12:35.991065 2834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.65:39937 every 8 connection(s)
I20240503 02:12:35.993431 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:12:36.016266 2839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:12:36.017132 2840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:36.021673 2835 heartbeater.cc:347] Connected to a master server at 127.2.157.126:34485
I20240503 02:12:36.022008 2835 heartbeater.cc:464] Registering TS with master...
I20240503 02:12:36.023339 2835 heartbeater.cc:511] Master 127.2.157.126:34485 requested a full tablet report, sending...
I20240503 02:12:36.025949 2707 ts_manager.cc:194] Registered new tserver with Master: 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937)
I20240503 02:12:36.026569 2677 server_base.cc:1034] running on GCE node
I20240503 02:12:36.027673 2707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59112
W20240503 02:12:36.029793 2843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:36.030229 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:12:36.030306 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:12:36.030368 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702356030328 us; error 0 us; skew 500 ppm
I20240503 02:12:36.030530 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:12:36.032179 2677 webserver.cc:460] Webserver started at http://127.2.157.66:40139/ using document root <none> and password file <none>
I20240503 02:12:36.032423 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:12:36.032526 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:12:36.032676 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:12:36.033386 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-1-root/instance:
uuid: "e74960ef531445d9a285c8c8802ec5c6"
format_stamp: "Formatted at 2024-05-03 02:12:36 on dist-test-slave-tz5s"
I20240503 02:12:36.036144 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20240503 02:12:36.038090 2847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:36.038522 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240503 02:12:36.038671 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-1-root
uuid: "e74960ef531445d9a285c8c8802ec5c6"
format_stamp: "Formatted at 2024-05-03 02:12:36 on dist-test-slave-tz5s"
I20240503 02:12:36.038831 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:12:36.057734 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:12:36.058730 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:12:36.059520 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:12:36.060720 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:12:36.060817 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:36.060947 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:12:36.061017 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:36.081670 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.66:39167
I20240503 02:12:36.081725 2909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.66:39167 every 8 connection(s)
I20240503 02:12:36.083660 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:12:36.087672 2913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:36.091665 2910 heartbeater.cc:347] Connected to a master server at 127.2.157.126:34485
W20240503 02:12:36.091748 2914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:36.091884 2910 heartbeater.cc:464] Registering TS with master...
I20240503 02:12:36.092568 2910 heartbeater.cc:511] Master 127.2.157.126:34485 requested a full tablet report, sending...
I20240503 02:12:36.093953 2707 ts_manager.cc:194] Registered new tserver with Master: e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:36.095325 2707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59120
I20240503 02:12:36.099362 2677 server_base.cc:1034] running on GCE node
W20240503 02:12:36.099993 2916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:12:36.100414 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:12:36.100525 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:12:36.100592 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702356100568 us; error 0 us; skew 500 ppm
I20240503 02:12:36.100778 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:12:36.102417 2677 webserver.cc:460] Webserver started at http://127.2.157.67:40153/ using document root <none> and password file <none>
I20240503 02:12:36.102659 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:12:36.102766 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:12:36.102934 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:12:36.103683 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-2-root/instance:
uuid: "1e80fe7edb59410e8c42c80d7bb325be"
format_stamp: "Formatted at 2024-05-03 02:12:36 on dist-test-slave-tz5s"
I20240503 02:12:36.106673 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240503 02:12:36.108649 2921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:36.109053 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240503 02:12:36.109185 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-2-root
uuid: "1e80fe7edb59410e8c42c80d7bb325be"
format_stamp: "Formatted at 2024-05-03 02:12:36 on dist-test-slave-tz5s"
I20240503 02:12:36.109306 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1714702355592969-2677-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:12:36.118860 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:12:36.119513 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:12:36.120155 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:12:36.121433 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:12:36.121570 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:36.121702 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:12:36.121785 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:12:36.143429 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.67:39391
I20240503 02:12:36.143496 2983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.67:39391 every 8 connection(s)
I20240503 02:12:36.152467 2984 heartbeater.cc:347] Connected to a master server at 127.2.157.126:34485
I20240503 02:12:36.152609 2984 heartbeater.cc:464] Registering TS with master...
I20240503 02:12:36.153123 2984 heartbeater.cc:511] Master 127.2.157.126:34485 requested a full tablet report, sending...
I20240503 02:12:36.154068 2707 ts_manager.cc:194] Registered new tserver with Master: 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.154820 2707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59126
I20240503 02:12:36.155006 2677 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01035843s
I20240503 02:12:36.156277 2677 test_util.cc:268] Using random seed: 1087737688
I20240503 02:12:36.180011 2707 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59134:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:12:36.182317 2707 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:12:36.225308 2949 tablet_service.cc:1467] Processing CreateTablet for tablet c77fed67fefe4d19a26f3aa1ce5faa58 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240503 02:12:36.227069 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77fed67fefe4d19a26f3aa1ce5faa58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.228178 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 9f88e54e477d436184ce7cbed290e5c3 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240503 02:12:36.228780 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f88e54e477d436184ce7cbed290e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.230075 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 36a389246b014357973c5be29652951b (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240503 02:12:36.230602 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a389246b014357973c5be29652951b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.231169 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 148e7dde40e94ea3b153d3d168e47c2c (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240503 02:12:36.231583 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 148e7dde40e94ea3b153d3d168e47c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.237102 2946 tablet_service.cc:1467] Processing CreateTablet for tablet d68962f2fc7b43c19a544ab38811a430 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240503 02:12:36.237712 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68962f2fc7b43c19a544ab38811a430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.239610 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb2742d409147deb30b47878472efbd (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240503 02:12:36.239842 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 88c66cd2f65840209c29997708cee755 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240503 02:12:36.241940 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c66cd2f65840209c29997708cee755. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.244217 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb2742d409147deb30b47878472efbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.244197 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 9b22bde32a4c41a7a60cf167ff1c4a43 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240503 02:12:36.251921 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b22bde32a4c41a7a60cf167ff1c4a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.250736 2800 tablet_service.cc:1467] Processing CreateTablet for tablet c77fed67fefe4d19a26f3aa1ce5faa58 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240503 02:12:36.252912 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 41cb7656a4914a5b843598decbaa1fd0 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240503 02:12:36.253541 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb2742d409147deb30b47878472efbd (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240503 02:12:36.254218 2799 tablet_service.cc:1467] Processing CreateTablet for tablet 9f88e54e477d436184ce7cbed290e5c3 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240503 02:12:36.256654 2875 tablet_service.cc:1467] Processing CreateTablet for tablet c77fed67fefe4d19a26f3aa1ce5faa58 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20240503 02:12:36.257493 2798 tablet_service.cc:1467] Processing CreateTablet for tablet 36a389246b014357973c5be29652951b (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240503 02:12:36.257565 2797 tablet_service.cc:1467] Processing CreateTablet for tablet d68962f2fc7b43c19a544ab38811a430 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240503 02:12:36.258436 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77fed67fefe4d19a26f3aa1ce5faa58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.258493 2873 tablet_service.cc:1467] Processing CreateTablet for tablet 36a389246b014357973c5be29652951b (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20240503 02:12:36.258878 2999 tablet_bootstrap.cc:492] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.258775 2871 tablet_service.cc:1467] Processing CreateTablet for tablet 148e7dde40e94ea3b153d3d168e47c2c (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240503 02:12:36.258814 2793 tablet_service.cc:1467] Processing CreateTablet for tablet 9b22bde32a4c41a7a60cf167ff1c4a43 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240503 02:12:36.256862 2795 tablet_service.cc:1467] Processing CreateTablet for tablet 88c66cd2f65840209c29997708cee755 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240503 02:12:36.260901 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41cb7656a4914a5b843598decbaa1fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.260039 2792 tablet_service.cc:1467] Processing CreateTablet for tablet 41cb7656a4914a5b843598decbaa1fd0 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240503 02:12:36.259485 2874 tablet_service.cc:1467] Processing CreateTablet for tablet 9f88e54e477d436184ce7cbed290e5c3 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20240503 02:12:36.259156 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 148e7dde40e94ea3b153d3d168e47c2c (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20240503 02:12:36.262255 2799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f88e54e477d436184ce7cbed290e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.260115 2872 tablet_service.cc:1467] Processing CreateTablet for tablet d68962f2fc7b43c19a544ab38811a430 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20240503 02:12:36.264034 2875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77fed67fefe4d19a26f3aa1ce5faa58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.265715 2870 tablet_service.cc:1467] Processing CreateTablet for tablet 88c66cd2f65840209c29997708cee755 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20240503 02:12:36.267133 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68962f2fc7b43c19a544ab38811a430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.267154 2871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 148e7dde40e94ea3b153d3d168e47c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.267359 2869 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb2742d409147deb30b47878472efbd (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20240503 02:12:36.277303 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 148e7dde40e94ea3b153d3d168e47c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.278970 2874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f88e54e477d436184ce7cbed290e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.276759 2868 tablet_service.cc:1467] Processing CreateTablet for tablet 9b22bde32a4c41a7a60cf167ff1c4a43 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20240503 02:12:36.278982 2867 tablet_service.cc:1467] Processing CreateTablet for tablet 41cb7656a4914a5b843598decbaa1fd0 (DEFAULT_TABLE table=test-workload [id=ec0e191cf37f41548946f0331e451af5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20240503 02:12:36.282069 2793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b22bde32a4c41a7a60cf167ff1c4a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.284022 2869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb2742d409147deb30b47878472efbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.285656 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c66cd2f65840209c29997708cee755. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.293931 2870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88c66cd2f65840209c29997708cee755. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.294350 2872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68962f2fc7b43c19a544ab38811a430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.294620 2868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b22bde32a4c41a7a60cf167ff1c4a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.294970 2867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41cb7656a4914a5b843598decbaa1fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.295500 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a389246b014357973c5be29652951b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.296380 2873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a389246b014357973c5be29652951b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.301374 2792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41cb7656a4914a5b843598decbaa1fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.307898 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb2742d409147deb30b47878472efbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:12:36.308074 2999 tablet_bootstrap.cc:654] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.316002 3000 tablet_bootstrap.cc:492] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.354553 2999 tablet_bootstrap.cc:492] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.354800 2999 ts_tablet_manager.cc:1367] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.100s user 0.001s sys 0.011s
I20240503 02:12:36.363344 3002 tablet_bootstrap.cc:492] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.364425 2999 raft_consensus.cc:348] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.364996 2999 raft_consensus.cc:374] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.365785 2999 raft_consensus.cc:729] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.367843 2999 consensus_queue.cc:260] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.376555 3000 tablet_bootstrap.cc:654] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.380519 2999 ts_tablet_manager.cc:1398] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.025s user 0.003s sys 0.011s
I20240503 02:12:36.380918 2984 heartbeater.cc:503] Master 127.2.157.126:34485 was elected leader, sending a full tablet report...
I20240503 02:12:36.381072 2999 tablet_bootstrap.cc:492] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.383405 2999 tablet_bootstrap.cc:654] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.383533 3002 tablet_bootstrap.cc:654] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.385565 3000 tablet_bootstrap.cc:492] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.385830 3000 ts_tablet_manager.cc:1367] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.072s user 0.008s sys 0.000s
I20240503 02:12:36.387382 3000 raft_consensus.cc:348] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.387853 3000 raft_consensus.cc:374] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.392729 3002 tablet_bootstrap.cc:492] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.393770 3002 ts_tablet_manager.cc:1367] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.006s
I20240503 02:12:36.394562 2999 tablet_bootstrap.cc:492] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.394728 2999 ts_tablet_manager.cc:1367] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20240503 02:12:36.394817 3002 raft_consensus.cc:348] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.393501 3000 raft_consensus.cc:729] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
W20240503 02:12:36.395076 2985 tablet.cc:2369] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 02:12:36.395083 3002 raft_consensus.cc:374] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.395412 3002 raft_consensus.cc:729] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.395570 2999 raft_consensus.cc:348] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.395792 2999 raft_consensus.cc:374] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.395900 2999 raft_consensus.cc:729] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.395830 3002 consensus_queue.cc:260] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.396164 2999 consensus_queue.cc:260] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.396620 2999 ts_tablet_manager.cc:1398] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:12:36.397127 2999 tablet_bootstrap.cc:492] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.398949 2999 tablet_bootstrap.cc:654] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.397145 3000 consensus_queue.cc:260] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.403429 3002 ts_tablet_manager.cc:1398] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20240503 02:12:36.403424 2910 heartbeater.cc:503] Master 127.2.157.126:34485 was elected leader, sending a full tablet report...
I20240503 02:12:36.403780 3002 tablet_bootstrap.cc:492] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.406807 3002 tablet_bootstrap.cc:654] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.414644 2835 heartbeater.cc:503] Master 127.2.157.126:34485 was elected leader, sending a full tablet report...
I20240503 02:12:36.414729 2999 tablet_bootstrap.cc:492] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.414904 2999 ts_tablet_manager.cc:1367] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.006s
I20240503 02:12:36.415416 3002 tablet_bootstrap.cc:492] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.415607 3002 ts_tablet_manager.cc:1367] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.001s
I20240503 02:12:36.416332 2999 raft_consensus.cc:348] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.416604 2999 raft_consensus.cc:374] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.416821 3002 raft_consensus.cc:348] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.417670 3000 ts_tablet_manager.cc:1398] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.030s user 0.000s sys 0.010s
I20240503 02:12:36.418061 2999 raft_consensus.cc:729] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.418123 3000 tablet_bootstrap.cc:492] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.418373 2999 consensus_queue.cc:260] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.418970 2999 ts_tablet_manager.cc:1398] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20240503 02:12:36.419288 2999 tablet_bootstrap.cc:492] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.421099 2999 tablet_bootstrap.cc:654] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.422017 3002 raft_consensus.cc:374] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.422118 3002 raft_consensus.cc:729] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.422364 3002 consensus_queue.cc:260] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.422819 3002 ts_tablet_manager.cc:1398] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20240503 02:12:36.423079 3002 tablet_bootstrap.cc:492] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.423274 3000 tablet_bootstrap.cc:654] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.424592 3002 tablet_bootstrap.cc:654] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.431411 3000 tablet_bootstrap.cc:492] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.431622 3000 ts_tablet_manager.cc:1367] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20240503 02:12:36.432384 3000 raft_consensus.cc:348] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.432596 3000 raft_consensus.cc:374] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.432686 3000 raft_consensus.cc:729] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.432950 3000 consensus_queue.cc:260] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.434032 3000 ts_tablet_manager.cc:1398] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:12:36.434358 3000 tablet_bootstrap.cc:492] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.435119 2999 tablet_bootstrap.cc:492] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.435341 2999 ts_tablet_manager.cc:1367] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.001s
I20240503 02:12:36.436371 2999 raft_consensus.cc:348] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.436674 2999 raft_consensus.cc:374] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.436789 2999 raft_consensus.cc:729] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.436921 3000 tablet_bootstrap.cc:654] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.437155 2999 consensus_queue.cc:260] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.437964 2999 ts_tablet_manager.cc:1398] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240503 02:12:36.438347 2999 tablet_bootstrap.cc:492] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.441589 3002 tablet_bootstrap.cc:492] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.441740 3002 ts_tablet_manager.cc:1367] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.014s
I20240503 02:12:36.443454 2999 tablet_bootstrap.cc:654] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.444463 3002 raft_consensus.cc:348] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.444672 3002 raft_consensus.cc:374] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.444753 3002 raft_consensus.cc:729] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.444964 3002 consensus_queue.cc:260] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.445575 3002 ts_tablet_manager.cc:1398] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:12:36.445906 3002 tablet_bootstrap.cc:492] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.446648 3000 tablet_bootstrap.cc:492] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.446836 3000 ts_tablet_manager.cc:1367] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.002s
I20240503 02:12:36.447669 3002 tablet_bootstrap.cc:654] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.447849 3000 raft_consensus.cc:348] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.448081 3000 raft_consensus.cc:374] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.448211 3000 raft_consensus.cc:729] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.448474 3000 consensus_queue.cc:260] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.449064 3000 ts_tablet_manager.cc:1398] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:12:36.449386 3000 tablet_bootstrap.cc:492] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.451330 3000 tablet_bootstrap.cc:654] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.455484 3009 raft_consensus.cc:482] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.455642 3009 raft_consensus.cc:504] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.456419 3002 tablet_bootstrap.cc:492] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.456610 3002 ts_tablet_manager.cc:1367] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20240503 02:12:36.458694 3009 leader_election.cc:288] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:36.458617 3002 raft_consensus.cc:348] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.458873 3002 raft_consensus.cc:374] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.458966 3002 raft_consensus.cc:729] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.459291 3002 consensus_queue.cc:260] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.459957 3002 ts_tablet_manager.cc:1398] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:12:36.460345 3002 tablet_bootstrap.cc:492] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.466955 3009 raft_consensus.cc:482] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.468753 3009 raft_consensus.cc:504] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.469784 3009 leader_election.cc:288] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.472362 3002 tablet_bootstrap.cc:654] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.478965 3000 tablet_bootstrap.cc:492] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.479167 3000 ts_tablet_manager.cc:1367] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.009s
I20240503 02:12:36.480031 3000 raft_consensus.cc:348] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.480273 3000 raft_consensus.cc:374] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.480376 3000 raft_consensus.cc:729] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.481643 2999 tablet_bootstrap.cc:492] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.481909 2999 ts_tablet_manager.cc:1367] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.016s
I20240503 02:12:36.481788 3000 consensus_queue.cc:260] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.482642 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.483261 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.483503 3000 ts_tablet_manager.cc:1398] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:12:36.483700 2959 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.483850 3000 tablet_bootstrap.cc:492] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.484211 3003 raft_consensus.cc:482] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.484365 3003 raft_consensus.cc:504] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.485594 3003 leader_election.cc:288] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:36.482923 2999 raft_consensus.cc:348] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.486747 2999 raft_consensus.cc:374] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.487602 3000 tablet_bootstrap.cc:654] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.487547 2775 leader_election.cc:302] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:36.487491 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.487830 2884 raft_consensus.cc:2454] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.487871 3009 raft_consensus.cc:2781] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.487949 3009 raft_consensus.cc:482] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.488062 3009 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.488343 2999 raft_consensus.cc:729] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.488479 2775 leader_election.cc:302] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.488641 3002 tablet_bootstrap.cc:492] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.488798 3002 ts_tablet_manager.cc:1367] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.005s
I20240503 02:12:36.488639 2999 consensus_queue.cc:260] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.490062 2958 raft_consensus.cc:2454] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.489667 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.494802 2885 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.501627 2999 ts_tablet_manager.cc:1398] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20240503 02:12:36.502072 2999 tablet_bootstrap.cc:492] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.503644 3020 raft_consensus.cc:2781] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.503808 3020 raft_consensus.cc:482] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.504317 3002 raft_consensus.cc:348] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.506304 3002 raft_consensus.cc:374] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.506659 3020 raft_consensus.cc:3037] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.506408 3009 raft_consensus.cc:504] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.508623 3002 raft_consensus.cc:729] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.509310 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.509199 3009 leader_election.cc:288] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:36.509927 2885 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.509750 3002 consensus_queue.cc:260] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.511946 3002 ts_tablet_manager.cc:1398] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.023s user 0.000s sys 0.003s
I20240503 02:12:36.512444 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.512651 2958 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.512915 3002 tablet_bootstrap.cc:492] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.520954 2885 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.522457 2775 leader_election.cc:302] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.522517 2958 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.523828 3009 raft_consensus.cc:2781] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.524382 2999 tablet_bootstrap.cc:654] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.524538 3002 tablet_bootstrap.cc:654] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.527037 3000 tablet_bootstrap.cc:492] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.527344 3000 ts_tablet_manager.cc:1367] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.006s
I20240503 02:12:36.527267 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.527992 3009 raft_consensus.cc:686] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
W20240503 02:12:36.528702 2924 leader_election.cc:341] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:12:36.528738 3009 consensus_queue.cc:237] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:36.531085 3000 raft_consensus.cc:348] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.531309 3000 raft_consensus.cc:374] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.531410 3000 raft_consensus.cc:729] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.531719 3000 consensus_queue.cc:260] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.531800 3020 raft_consensus.cc:504] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.532538 3020 leader_election.cc:288] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.532321 3000 ts_tablet_manager.cc:1398] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240503 02:12:36.532950 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.533062 3000 tablet_bootstrap.cc:492] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.533151 2958 raft_consensus.cc:3037] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.533421 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.533733 2885 raft_consensus.cc:3037] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.535128 3000 tablet_bootstrap.cc:654] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.536113 2958 raft_consensus.cc:2454] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.536573 2775 leader_election.cc:302] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:36.536878 3023 raft_consensus.cc:2781] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.537039 3023 raft_consensus.cc:686] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:36.537339 3023 consensus_queue.cc:237] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.537963 2885 raft_consensus.cc:2454] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.541204 3000 tablet_bootstrap.cc:492] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.541384 3000 ts_tablet_manager.cc:1367] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20240503 02:12:36.541666 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0906dca094bd43a190401b5abde7bd21" is_pre_election: true
I20240503 02:12:36.541949 2999 tablet_bootstrap.cc:492] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.542147 2999 ts_tablet_manager.cc:1367] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.002s
I20240503 02:12:36.542177 3000 raft_consensus.cc:348] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
W20240503 02:12:36.542413 2922 leader_election.cc:341] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:12:36.542543 2922 leader_election.cc:302] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be; no voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:36.542791 3003 raft_consensus.cc:2726] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:12:36.542937 3000 raft_consensus.cc:374] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.543081 3000 raft_consensus.cc:729] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.543386 3000 consensus_queue.cc:260] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.543867 3000 ts_tablet_manager.cc:1398] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:12:36.543973 2999 raft_consensus.cc:348] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.544117 3000 tablet_bootstrap.cc:492] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.544214 2999 raft_consensus.cc:374] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.544323 2999 raft_consensus.cc:729] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.544597 2999 consensus_queue.cc:260] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.545686 3000 tablet_bootstrap.cc:654] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.545846 2999 ts_tablet_manager.cc:1398] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20240503 02:12:36.546178 2999 tablet_bootstrap.cc:492] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.548017 2999 tablet_bootstrap.cc:654] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.554432 3002 tablet_bootstrap.cc:492] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.554641 3002 ts_tablet_manager.cc:1367] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.007s
I20240503 02:12:36.554739 3000 tablet_bootstrap.cc:492] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.554909 3000 ts_tablet_manager.cc:1367] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20240503 02:12:36.555574 3002 raft_consensus.cc:348] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.555801 3002 raft_consensus.cc:374] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.555917 3002 raft_consensus.cc:729] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.556128 2999 tablet_bootstrap.cc:492] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.556258 2999 ts_tablet_manager.cc:1367] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20240503 02:12:36.556185 3002 consensus_queue.cc:260] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.556754 3002 ts_tablet_manager.cc:1398] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:12:36.557072 3002 tablet_bootstrap.cc:492] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.555828 3000 raft_consensus.cc:348] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.558817 3000 raft_consensus.cc:374] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.559353 3002 tablet_bootstrap.cc:654] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.559955 3000 raft_consensus.cc:729] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.560056 2999 raft_consensus.cc:348] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.560173 3000 consensus_queue.cc:260] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.560736 3000 ts_tablet_manager.cc:1398] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20240503 02:12:36.559266 2705 catalog_manager.cc:5484] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.561050 3000 tablet_bootstrap.cc:492] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.562698 3000 tablet_bootstrap.cc:654] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.563634 2999 raft_consensus.cc:374] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.564060 2999 raft_consensus.cc:729] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.564637 3002 tablet_bootstrap.cc:492] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.565397 3002 ts_tablet_manager.cc:1367] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20240503 02:12:36.564356 2999 consensus_queue.cc:260] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.565833 3000 tablet_bootstrap.cc:492] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.566012 3000 ts_tablet_manager.cc:1367] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:12:36.566197 2999 ts_tablet_manager.cc:1398] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20240503 02:12:36.566572 2999 tablet_bootstrap.cc:492] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.568323 2999 tablet_bootstrap.cc:654] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.568226 2707 catalog_manager.cc:5484] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.566870 3000 raft_consensus.cc:348] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.569411 3000 raft_consensus.cc:374] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.569540 3000 raft_consensus.cc:729] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.569844 3000 consensus_queue.cc:260] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.570354 3000 ts_tablet_manager.cc:1398] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:12:36.570351 3002 raft_consensus.cc:348] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.570693 3000 tablet_bootstrap.cc:492] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: Bootstrap starting.
I20240503 02:12:36.570716 3002 raft_consensus.cc:374] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.570925 3002 raft_consensus.cc:729] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.571216 3002 consensus_queue.cc:260] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.571964 3002 ts_tablet_manager.cc:1398] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:12:36.572327 3002 tablet_bootstrap.cc:492] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.574244 3000 tablet_bootstrap.cc:654] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.575436 3002 tablet_bootstrap.cc:654] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.578102 2999 tablet_bootstrap.cc:492] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.578269 2999 ts_tablet_manager.cc:1367] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20240503 02:12:36.578380 3000 tablet_bootstrap.cc:492] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: No bootstrap required, opened a new log
I20240503 02:12:36.578630 3000 ts_tablet_manager.cc:1367] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20240503 02:12:36.578984 2999 raft_consensus.cc:348] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.579186 2999 raft_consensus.cc:374] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.579275 2999 raft_consensus.cc:729] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.579519 2999 consensus_queue.cc:260] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.579981 2999 ts_tablet_manager.cc:1398] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:12:36.579996 3000 raft_consensus.cc:348] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.580283 3000 raft_consensus.cc:374] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.580513 2999 tablet_bootstrap.cc:492] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be: Bootstrap starting.
I20240503 02:12:36.580502 3000 raft_consensus.cc:729] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.581561 3000 consensus_queue.cc:260] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.582238 3000 ts_tablet_manager.cc:1398] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240503 02:12:36.583323 2999 tablet_bootstrap.cc:654] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.584889 3002 tablet_bootstrap.cc:492] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.585085 3002 ts_tablet_manager.cc:1367] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20240503 02:12:36.585494 2999 tablet_bootstrap.cc:492] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be: No bootstrap required, opened a new log
I20240503 02:12:36.585646 2999 ts_tablet_manager.cc:1367] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240503 02:12:36.585886 3002 raft_consensus.cc:348] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.586162 3002 raft_consensus.cc:374] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.586270 3002 raft_consensus.cc:729] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.586185 2999 raft_consensus.cc:348] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.586411 2999 raft_consensus.cc:374] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.586619 2999 raft_consensus.cc:729] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.586565 3002 consensus_queue.cc:260] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.586867 2999 consensus_queue.cc:260] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.587070 3002 ts_tablet_manager.cc:1398] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:12:36.587415 2999 ts_tablet_manager.cc:1398] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:12:36.587870 3002 tablet_bootstrap.cc:492] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: Bootstrap starting.
I20240503 02:12:36.588917 3007 raft_consensus.cc:482] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.589066 3007 raft_consensus.cc:504] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.589557 3002 tablet_bootstrap.cc:654] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: Neither blocks nor log segments found. Creating new log.
I20240503 02:12:36.590359 3007 leader_election.cc:288] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.601179 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.601434 2958 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 0.
I20240503 02:12:36.601811 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0906dca094bd43a190401b5abde7bd21" is_pre_election: true
I20240503 02:12:36.601817 2850 leader_election.cc:302] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.602061 2810 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 0.
I20240503 02:12:36.602239 3007 raft_consensus.cc:2781] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.602434 3007 raft_consensus.cc:482] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.602582 3007 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.603655 3002 tablet_bootstrap.cc:492] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: No bootstrap required, opened a new log
I20240503 02:12:36.603799 3002 ts_tablet_manager.cc:1367] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20240503 02:12:36.604485 3002 raft_consensus.cc:348] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.604624 3002 raft_consensus.cc:374] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:12:36.604727 3002 raft_consensus.cc:729] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Initialized, Role: FOLLOWER
I20240503 02:12:36.604883 3007 raft_consensus.cc:504] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.604938 3002 consensus_queue.cc:260] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.605466 3007 leader_election.cc:288] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 1 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.605974 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.606153 2958 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.606195 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:36.606463 2810 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.609000 2958 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 1.
I20240503 02:12:36.609767 2850 leader_election.cc:302] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.610116 3007 raft_consensus.cc:2781] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.610252 3002 ts_tablet_manager.cc:1398] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20240503 02:12:36.610260 3007 raft_consensus.cc:686] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:36.610646 2810 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 1.
I20240503 02:12:36.610651 3007 consensus_queue.cc:237] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.613315 2705 catalog_manager.cc:5484] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.636204 3003 raft_consensus.cc:482] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.636413 3003 raft_consensus.cc:504] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.637017 3003 leader_election.cc:288] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:36.637468 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0906dca094bd43a190401b5abde7bd21" is_pre_election: true
I20240503 02:12:36.637496 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.637676 2810 raft_consensus.cc:2454] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 0.
I20240503 02:12:36.637799 2885 raft_consensus.cc:2454] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 0.
I20240503 02:12:36.638101 2922 leader_election.cc:302] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:36.638357 3003 raft_consensus.cc:2781] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.638471 3003 raft_consensus.cc:482] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.638577 3003 raft_consensus.cc:3037] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.640484 3003 raft_consensus.cc:504] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.641044 3003 leader_election.cc:288] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:36.641403 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:36.641423 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.641666 2885 raft_consensus.cc:3037] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.641644 2810 raft_consensus.cc:3037] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.644692 2885 raft_consensus.cc:2454] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 1.
I20240503 02:12:36.644693 2810 raft_consensus.cc:2454] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 1.
I20240503 02:12:36.645134 2924 leader_election.cc:302] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.645445 3003 raft_consensus.cc:2781] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.646004 3003 raft_consensus.cc:686] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 LEADER]: Becoming Leader. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:36.646376 3003 consensus_queue.cc:237] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.649118 2705 catalog_manager.cc:5484] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67). New cstate: current_term: 1 leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: HEALTHY } } }
I20240503 02:12:36.665385 3009 raft_consensus.cc:482] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.665609 3009 raft_consensus.cc:504] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.666206 3009 leader_election.cc:288] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.666782 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.666816 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.666981 2885 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.667069 2958 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.667506 2775 leader_election.cc:302] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.667791 3009 raft_consensus.cc:2781] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.667912 3009 raft_consensus.cc:482] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.668015 3009 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.669857 3009 raft_consensus.cc:504] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.670404 3009 leader_election.cc:288] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.670791 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.670857 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.670930 2885 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.671090 2958 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.672904 2885 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.673106 2958 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.673293 2775 leader_election.cc:302] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.673599 3009 raft_consensus.cc:2781] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.673744 3009 raft_consensus.cc:686] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:36.674005 3009 consensus_queue.cc:237] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.676507 2705 catalog_manager.cc:5484] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.687664 3009 raft_consensus.cc:482] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.687832 3009 raft_consensus.cc:504] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.688338 3009 leader_election.cc:288] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.688704 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.688750 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.688880 2885 raft_consensus.cc:2454] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.688947 2958 raft_consensus.cc:2454] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.689249 2775 leader_election.cc:302] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.689522 3009 raft_consensus.cc:2781] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.689641 3009 raft_consensus.cc:482] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.689738 3009 raft_consensus.cc:3037] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.691555 3009 raft_consensus.cc:504] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.692027 3009 leader_election.cc:288] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.692359 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.692430 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.692538 2885 raft_consensus.cc:3037] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.692582 2958 raft_consensus.cc:3037] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.694422 2885 raft_consensus.cc:2454] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.694464 2958 raft_consensus.cc:2454] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.694741 2775 leader_election.cc:302] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.694970 3009 raft_consensus.cc:2781] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.695140 3009 raft_consensus.cc:686] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:36.695392 3009 consensus_queue.cc:237] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.697647 2705 catalog_manager.cc:5484] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.779974 3009 raft_consensus.cc:482] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.780189 3009 raft_consensus.cc:504] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.780781 3009 leader_election.cc:288] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.781373 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.781355 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.781635 2958 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.781641 2885 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.781987 2775 leader_election.cc:302] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.782215 3009 raft_consensus.cc:2781] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.782315 3009 raft_consensus.cc:482] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.782464 3009 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.784271 3009 raft_consensus.cc:504] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.784781 3009 leader_election.cc:288] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.785274 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.785296 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.785460 2885 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.785499 2958 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.787634 2958 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.788017 2775 leader_election.cc:302] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:36.788210 2885 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.788285 3009 raft_consensus.cc:2781] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
W20240503 02:12:36.788316 2755 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 02:12:36.788445 3009 raft_consensus.cc:686] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:36.788736 3009 consensus_queue.cc:237] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.791126 2705 catalog_manager.cc:5484] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.836995 3009 raft_consensus.cc:482] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.837239 3009 raft_consensus.cc:504] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.837921 3009 leader_election.cc:288] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.838510 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.838506 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.838857 2885 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.838837 2958 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.839349 2775 leader_election.cc:302] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:36.839679 3009 raft_consensus.cc:2781] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.839810 3009 raft_consensus.cc:482] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.839932 3009 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.841938 3009 raft_consensus.cc:504] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.842542 3009 leader_election.cc:288] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.842873 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.842919 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.843093 2885 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.843091 2958 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.846181 2885 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.846184 2958 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.846583 2775 leader_election.cc:302] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.846841 3009 raft_consensus.cc:2781] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.847007 3009 raft_consensus.cc:686] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:36.847265 3009 consensus_queue.cc:237] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.849764 2705 catalog_manager.cc:5484] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:36.953711 3009 raft_consensus.cc:482] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:12:36.953894 3009 raft_consensus.cc:504] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.954556 3009 leader_election.cc:288] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.954928 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6" is_pre_election: true
I20240503 02:12:36.955031 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be" is_pre_election: true
I20240503 02:12:36.955125 2885 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.955245 2958 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 0.
I20240503 02:12:36.955546 2775 leader_election.cc:302] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.955767 3009 raft_consensus.cc:2781] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:12:36.955880 3009 raft_consensus.cc:482] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:12:36.955973 3009 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.957991 3009 raft_consensus.cc:504] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.958442 3009 leader_election.cc:288] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:36.958830 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:36.958854 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:36.958984 2885 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.959026 2958 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:12:36.960804 2885 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.960830 2958 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 1.
I20240503 02:12:36.961112 2775 leader_election.cc:302] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:36.961364 3009 raft_consensus.cc:2781] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:12:36.961562 3009 raft_consensus.cc:686] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:36.961825 3009 consensus_queue.cc:237] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:36.964334 2705 catalog_manager.cc:5484] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:37.003958 3009 consensus_queue.cc:1035] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.009337 3023 consensus_queue.cc:1035] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.039201 3003 consensus_queue.cc:1035] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.044602 3042 consensus_queue.cc:1035] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.082938 3023 consensus_queue.cc:1035] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.086182 3023 consensus_queue.cc:1035] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.088371 3020 consensus_queue.cc:1035] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.091279 3023 consensus_queue.cc:1035] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.135725 2958 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:45942
I20240503 02:12:37.135977 2958 raft_consensus.cc:595] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:37.139530 3009 consensus_queue.cc:1035] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.144588 2810 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "88c66cd2f65840209c29997708cee755"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:53808
I20240503 02:12:37.144786 2810 raft_consensus.cc:595] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:37.145476 3058 consensus_queue.cc:1035] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.147477 2810 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5fb2742d409147deb30b47878472efbd"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:53808
I20240503 02:12:37.147682 2810 raft_consensus.cc:595] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:37.151016 2810 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:53808
I20240503 02:12:37.151188 2810 raft_consensus.cc:595] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS 1e80fe7edb59410e8c42c80d7bb325be
I20240503 02:12:37.180094 3007 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.183470 3068 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.233815 3058 consensus_queue.cc:1035] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.238034 3023 consensus_queue.cc:1035] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.240931 3023 raft_consensus.cc:979] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:37.241135 3023 raft_consensus.cc:1067] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:37.242038 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5fb2742d409147deb30b47878472efbd"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39414
I20240503 02:12:37.242236 2885 raft_consensus.cc:482] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:37.242344 2885 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.244136 2885 raft_consensus.cc:504] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.244714 2885 leader_election.cc:288] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:37.245162 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:37.245393 2810 raft_consensus.cc:3032] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:37.245493 2810 raft_consensus.cc:729] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:37.245656 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:37.245853 2958 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.245857 2810 consensus_queue.cc:260] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.246435 2810 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.248195 2958 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:37.248488 2810 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:37.248538 2850 leader_election.cc:302] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:37.248873 3069 raft_consensus.cc:2781] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:37.249028 3069 raft_consensus.cc:686] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:37.249325 3069 consensus_queue.cc:237] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.251291 2705 catalog_manager.cc:5484] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:37.255303 3023 consensus_queue.cc:1035] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.259968 3023 consensus_queue.cc:1035] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.262230 3009 raft_consensus.cc:979] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:37.262410 3009 raft_consensus.cc:1067] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:37.263006 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "88c66cd2f65840209c29997708cee755"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39414
I20240503 02:12:37.263161 2885 raft_consensus.cc:482] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:37.263270 2885 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.264946 2885 raft_consensus.cc:504] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.265515 2885 leader_election.cc:288] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:37.265908 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:37.266072 2810 raft_consensus.cc:3032] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:37.266132 2810 raft_consensus.cc:729] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:37.266346 2810 consensus_queue.cc:260] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.266589 2810 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.268325 2810 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:37.268628 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:37.268684 2848 leader_election.cc:302] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:37.268810 2958 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.268956 3069 raft_consensus.cc:2781] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:37.269115 3069 raft_consensus.cc:686] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:37.269384 3069 consensus_queue.cc:237] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.271133 2958 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:37.271494 2707 catalog_manager.cc:5484] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:37.447042 3003 raft_consensus.cc:979] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:37.447234 3003 raft_consensus.cc:1067] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:37.447826 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39426
I20240503 02:12:37.448055 2885 raft_consensus.cc:482] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:37.448151 2885 raft_consensus.cc:3037] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.450110 2885 raft_consensus.cc:504] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.450654 2885 leader_election.cc:288] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:37.451067 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:37.451118 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "148e7dde40e94ea3b153d3d168e47c2c" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:37.451314 2958 raft_consensus.cc:3032] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:37.451313 2810 raft_consensus.cc:3037] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.451462 2958 raft_consensus.cc:729] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:37.451757 2958 consensus_queue.cc:260] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.452152 2958 raft_consensus.cc:3037] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.454371 2810 raft_consensus.cc:2454] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:37.454785 2848 leader_election.cc:302] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:37.454852 2958 raft_consensus.cc:2454] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:37.455088 3069 raft_consensus.cc:2781] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:37.455242 3069 raft_consensus.cc:686] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:37.455531 3069 consensus_queue.cc:237] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.457830 2707 catalog_manager.cc:5484] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 1 to 2, leader changed from 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:37.514951 3009 consensus_queue.cc:1035] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.519626 3023 consensus_queue.cc:1035] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:12:37.666725 3023 raft_consensus.cc:979] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: : Instructing follower 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:37.666949 3023 raft_consensus.cc:1067] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:37.667634 2958 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:45928
I20240503 02:12:37.667891 2958 raft_consensus.cc:482] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:37.668044 2958 raft_consensus.cc:3037] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.670017 2958 raft_consensus.cc:504] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.670591 2958 leader_election.cc:288] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:37.670899 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:37.671082 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cb7656a4914a5b843598decbaa1fd0" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:37.671136 2810 raft_consensus.cc:3032] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:37.671280 2884 raft_consensus.cc:3037] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.671344 2810 raft_consensus.cc:729] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:37.671620 2810 consensus_queue.cc:260] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.671999 2810 raft_consensus.cc:3037] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:37.674255 2884 raft_consensus.cc:2454] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 2.
I20240503 02:12:37.674674 2810 raft_consensus.cc:2454] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 2.
I20240503 02:12:37.674710 2924 leader_election.cc:302] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:37.674988 3003 raft_consensus.cc:2781] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:37.675137 3003 raft_consensus.cc:686] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Becoming Leader. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:37.675388 3003 consensus_queue.cc:237] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:37.677394 2707 catalog_manager.cc:5484] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67). New cstate: current_term: 2 leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: HEALTHY } } }
I20240503 02:12:37.703028 2810 raft_consensus.cc:1261] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:37.703557 3007 consensus_queue.cc:1035] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:37.706310 2958 raft_consensus.cc:1261] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:37.706948 3068 consensus_queue.cc:1035] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:37.723820 2810 raft_consensus.cc:1261] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:37.724354 3068 consensus_queue.cc:1035] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:37.727437 2958 raft_consensus.cc:1261] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:37.728019 3007 consensus_queue.cc:1035] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:37.959767 2958 raft_consensus.cc:1261] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:37.960331 3007 consensus_queue.cc:1035] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:37.963915 2810 raft_consensus.cc:1261] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:37.964396 3069 consensus_queue.cc:1035] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:38.096603 2884 raft_consensus.cc:1261] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:38.097296 3042 consensus_queue.cc:1035] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:38.102399 2810 raft_consensus.cc:1261] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:38.103494 3042 consensus_queue.cc:1035] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:40.160740 2677 auto_leader_rebalancer-test.cc:261] The leader distribution is
I20240503 02:12:40.161094 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:40.168843 2884 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
mode: GRACEFUL
new_leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:39432
I20240503 02:12:40.169011 2884 raft_consensus.cc:595] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 LEADER]: Received request to transfer leadership to TS 1e80fe7edb59410e8c42c80d7bb325be
I20240503 02:12:40.174476 2810 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:53824
I20240503 02:12:40.174625 2810 raft_consensus.cc:595] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS 1e80fe7edb59410e8c42c80d7bb325be
I20240503 02:12:40.175019 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240503 02:12:40.175240 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:40.292968 3023 raft_consensus.cc:979] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: : Instructing follower 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:40.293154 3023 raft_consensus.cc:1067] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:40.293804 2958 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:45928
I20240503 02:12:40.293999 2958 raft_consensus.cc:482] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:40.294111 2958 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:40.295948 2958 raft_consensus.cc:504] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:40.296577 2958 leader_election.cc:288] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:40.296988 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:40.297015 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:40.297152 2810 raft_consensus.cc:3032] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:40.297184 2884 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:40.297269 2810 raft_consensus.cc:729] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:40.297509 2810 consensus_queue.cc:260] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:40.297863 2810 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:40.299391 2884 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 2.
I20240503 02:12:40.299813 2924 leader_election.cc:302] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:40.300006 2810 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 2.
I20240503 02:12:40.300190 3082 raft_consensus.cc:2781] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:40.300686 3082 raft_consensus.cc:686] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Becoming Leader. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:40.300969 3082 consensus_queue.cc:237] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:40.303486 2707 catalog_manager.cc:5484] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67). New cstate: current_term: 2 leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: HEALTHY } } }
I20240503 02:12:40.329433 3007 raft_consensus.cc:979] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: : Instructing follower 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:40.329646 3007 raft_consensus.cc:1067] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 LEADER]: Signalling peer 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:40.330338 2958 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:45938
I20240503 02:12:40.330574 2958 raft_consensus.cc:482] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:40.330679 2958 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:40.332809 2958 raft_consensus.cc:504] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:40.333726 2958 leader_election.cc:288] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:40.333871 2810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:40.333921 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:40.334106 2810 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:40.334110 2884 raft_consensus.cc:3032] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:40.334229 2884 raft_consensus.cc:729] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:40.334501 2884 consensus_queue.cc:260] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:40.334939 2884 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:40.337133 2810 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 2.
I20240503 02:12:40.337561 2922 leader_election.cc:302] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:40.337694 2884 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 2.
I20240503 02:12:40.337904 3082 raft_consensus.cc:2781] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:40.338142 3082 raft_consensus.cc:686] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Becoming Leader. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:40.338429 3082 consensus_queue.cc:237] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:40.340862 2707 catalog_manager.cc:5484] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be reported cstate change: term changed from 1 to 2, leader changed from e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66) to 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67). New cstate: current_term: 2 leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: HEALTHY } } }
I20240503 02:12:40.719058 2884 raft_consensus.cc:1261] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:40.719609 3087 consensus_queue.cc:1035] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:40.723091 2810 raft_consensus.cc:1261] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:40.723631 3087 consensus_queue.cc:1035] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:40.877399 2810 raft_consensus.cc:1261] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:40.878015 3087 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:40.881399 2884 raft_consensus.cc:1261] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:40.882890 3101 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 02:12:41.175740 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:41.177305 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:41.177497 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:42.177896 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:42.179454 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:42.179627 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:43.180079 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:43.182030 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:43.182229 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:44.182616 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:44.184213 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:44.184412 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:45.184841 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:45.186666 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:45.186873 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:46.187220 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:46.188803 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:46.188973 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:47.189360 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:47.191165 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:47.191356 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:48.191735 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:48.193692 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:48.193857 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:49.194295 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:49.195822 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:49.196003 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:50.206282 2677 auto_leader_rebalancer-test.cc:280] The leader distribution is
I20240503 02:12:50.207983 2809 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:53808
I20240503 02:12:50.208215 2809 raft_consensus.cc:595] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:50.210109 2958 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:45942
I20240503 02:12:50.210304 2958 raft_consensus.cc:595] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:50.212246 2809 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "36a389246b014357973c5be29652951b"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:53808
I20240503 02:12:50.212388 2809 raft_consensus.cc:595] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:50.214212 2809 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:53808
I20240503 02:12:50.214344 2809 raft_consensus.cc:595] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:50.219018 2958 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
mode: GRACEFUL
new_leader_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:45942
I20240503 02:12:50.219197 2958 raft_consensus.cc:595] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Received request to transfer leadership to TS e74960ef531445d9a285c8c8802ec5c6
I20240503 02:12:50.256093 3023 raft_consensus.cc:979] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.256292 3023 raft_consensus.cc:1067] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.256927 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39414
I20240503 02:12:50.257071 2885 raft_consensus.cc:482] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:50.257189 2885 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.259159 2885 raft_consensus.cc:504] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:50.259683 2885 leader_election.cc:288] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:50.260179 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:50.260170 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:50.260387 2809 raft_consensus.cc:3032] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:50.260390 2959 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.260516 2809 raft_consensus.cc:729] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:50.260812 2809 consensus_queue.cc:260] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:50.261202 2809 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.263486 2959 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:50.263955 2809 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:50.264004 2850 leader_election.cc:302] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:50.264365 3007 raft_consensus.cc:2781] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:50.264775 3007 raft_consensus.cc:686] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:50.265148 3007 consensus_queue.cc:237] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:50.267958 2705 catalog_manager.cc:5484] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } }
I20240503 02:12:50.499445 3103 raft_consensus.cc:979] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.499624 3103 raft_consensus.cc:1067] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.500226 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "36a389246b014357973c5be29652951b"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39414
I20240503 02:12:50.500409 2885 raft_consensus.cc:482] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:50.500490 2885 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.502404 2885 raft_consensus.cc:504] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.502960 2885 leader_election.cc:288] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:50.503386 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:50.503413 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:50.503551 2809 raft_consensus.cc:3032] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:50.503612 2959 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.503681 2809 raft_consensus.cc:729] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:50.503969 2809 consensus_queue.cc:260] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.504269 2809 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.505761 2959 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:50.506145 2850 leader_election.cc:302] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80fe7edb59410e8c42c80d7bb325be, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:50.506248 2809 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:50.506458 3007 raft_consensus.cc:2781] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:50.506598 3007 raft_consensus.cc:686] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:50.506860 3007 consensus_queue.cc:237] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.509426 2705 catalog_manager.cc:5484] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:50.526607 3101 raft_consensus.cc:979] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.526772 3101 raft_consensus.cc:1067] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.527356 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39426
I20240503 02:12:50.527506 2885 raft_consensus.cc:482] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:50.527601 2885 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:50.529333 2885 raft_consensus.cc:504] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.529853 2885 leader_election.cc:288] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 3 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:50.530187 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:50.530265 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f88e54e477d436184ce7cbed290e5c3" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:50.530391 2809 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:50.530395 2959 raft_consensus.cc:3032] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:12:50.530517 2959 raft_consensus.cc:729] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:50.530776 2959 consensus_queue.cc:260] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.531220 2959 raft_consensus.cc:3037] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:50.533329 2809 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 3.
I20240503 02:12:50.533753 2848 leader_election.cc:302] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:50.533849 2959 raft_consensus.cc:2454] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 3.
I20240503 02:12:50.534030 3007 raft_consensus.cc:2781] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:12:50.534210 3007 raft_consensus.cc:686] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:50.534466 3007 consensus_queue.cc:237] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.536759 2705 catalog_manager.cc:5484] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 2 to 3, leader changed from 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 3 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:50.717123 2809 raft_consensus.cc:1261] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:50.717837 3007 consensus_queue.cc:1035] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:50.723263 2959 raft_consensus.cc:1261] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:50.723418 3101 raft_consensus.cc:979] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.723590 3101 raft_consensus.cc:1067] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.724030 3115 consensus_queue.cc:1035] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:50.724530 2885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39426
I20240503 02:12:50.724786 2885 raft_consensus.cc:482] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:50.724959 2885 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:50.727749 3023 raft_consensus.cc:979] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: : Instructing follower e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.727916 3103 raft_consensus.cc:1067] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Signalling peer e74960ef531445d9a285c8c8802ec5c6 to start an election
I20240503 02:12:50.728519 2884 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
from {username='slave'} at 127.0.0.1:39414
I20240503 02:12:50.728736 2884 raft_consensus.cc:482] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:50.728863 2884 raft_consensus.cc:3037] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.729053 2885 raft_consensus.cc:504] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.730250 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:50.730496 2809 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:50.731483 2884 raft_consensus.cc:504] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.732197 2884 leader_election.cc:288] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:50.732861 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:50.733057 2959 raft_consensus.cc:3037] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.733399 2809 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 3.
I20240503 02:12:50.733881 2848 leader_election.cc:302] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:50.734126 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68962f2fc7b43c19a544ab38811a430" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:50.734197 3115 raft_consensus.cc:2781] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:12:50.734568 2809 raft_consensus.cc:3032] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:12:50.734681 2958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "e74960ef531445d9a285c8c8802ec5c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:50.734760 3115 raft_consensus.cc:686] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:50.734863 2958 raft_consensus.cc:3032] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:12:50.734565 2885 leader_election.cc:288] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 3 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:50.734968 2958 raft_consensus.cc:729] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:50.735119 3115 consensus_queue.cc:237] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.735284 2958 consensus_queue.cc:260] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.735651 2958 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:50.734762 2809 raft_consensus.cc:729] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:50.736733 2809 consensus_queue.cc:260] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.737205 2809 raft_consensus.cc:3037] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:12:50.738243 2958 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 3.
I20240503 02:12:50.739405 2707 catalog_manager.cc:5484] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 2 to 3, leader changed from 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 3 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:50.739820 2809 raft_consensus.cc:2454] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:50.740260 2848 leader_election.cc:302] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, e74960ef531445d9a285c8c8802ec5c6; no voters:
I20240503 02:12:50.740319 2959 raft_consensus.cc:2454] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e74960ef531445d9a285c8c8802ec5c6 in term 2.
I20240503 02:12:50.740567 3115 raft_consensus.cc:2781] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:12:50.740696 3115 raft_consensus.cc:686] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Leader. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:50.740880 3115 consensus_queue.cc:237] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:50.744009 2707 catalog_manager.cc:5484] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 reported cstate change: term changed from 1 to 2, leader changed from 0906dca094bd43a190401b5abde7bd21 (127.2.157.65) to e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "e74960ef531445d9a285c8c8802ec5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:50.899755 2959 raft_consensus.cc:1261] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:50.900239 3115 consensus_queue.cc:1035] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:50.903540 2809 raft_consensus.cc:1261] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:50.904132 3115 consensus_queue.cc:1035] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:51.041457 2959 raft_consensus.cc:1261] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:51.042819 3115 consensus_queue.cc:1035] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:51.046295 2809 raft_consensus.cc:1261] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:51.046775 3007 consensus_queue.cc:1035] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:51.194145 2810 raft_consensus.cc:1261] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:51.194703 3115 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:51.198056 2959 raft_consensus.cc:1261] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:51.198843 3115 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:51.243943 2810 raft_consensus.cc:1261] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:51.244500 3115 consensus_queue.cc:1035] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:51.248711 2959 raft_consensus.cc:1261] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Refusing update from remote peer e74960ef531445d9a285c8c8802ec5c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:12:51.249222 3007 consensus_queue.cc:1035] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:12:53.230593 2677 auto_leader_rebalancer-test.cc:295] The leader distribution is
I20240503 02:12:53.230860 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:53.233094 2885 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "36a389246b014357973c5be29652951b"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
mode: GRACEFUL
new_leader_uuid: "0906dca094bd43a190401b5abde7bd21"
from {username='slave'} at 127.0.0.1:39432
I20240503 02:12:53.233302 2885 raft_consensus.cc:595] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Received request to transfer leadership to TS 0906dca094bd43a190401b5abde7bd21
I20240503 02:12:53.234477 2885 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5fb2742d409147deb30b47878472efbd"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
mode: GRACEFUL
new_leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:39432
I20240503 02:12:53.234679 2885 raft_consensus.cc:595] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Received request to transfer leadership to TS 1e80fe7edb59410e8c42c80d7bb325be
I20240503 02:12:53.235647 2885 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "88c66cd2f65840209c29997708cee755"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
mode: GRACEFUL
new_leader_uuid: "0906dca094bd43a190401b5abde7bd21"
from {username='slave'} at 127.0.0.1:39432
I20240503 02:12:53.235787 2885 raft_consensus.cc:595] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Received request to transfer leadership to TS 0906dca094bd43a190401b5abde7bd21
I20240503 02:12:53.236723 2885 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
mode: GRACEFUL
new_leader_uuid: "0906dca094bd43a190401b5abde7bd21"
from {username='slave'} at 127.0.0.1:39432
I20240503 02:12:53.236866 2885 raft_consensus.cc:595] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Received request to transfer leadership to TS 0906dca094bd43a190401b5abde7bd21
I20240503 02:12:53.237744 2885 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58"
dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
mode: GRACEFUL
new_leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:39432
I20240503 02:12:53.237882 2885 raft_consensus.cc:595] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Received request to transfer leadership to TS 1e80fe7edb59410e8c42c80d7bb325be
I20240503 02:12:53.238234 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 5
I20240503 02:12:53.238426 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:53.270285 3131 raft_consensus.cc:979] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: : Instructing follower 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:53.270452 3131 raft_consensus.cc:1067] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Signalling peer 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:53.271080 2959 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5fb2742d409147deb30b47878472efbd"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:45938
I20240503 02:12:53.271272 2959 raft_consensus.cc:482] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:53.271420 2959 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.274067 2959 raft_consensus.cc:504] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.275076 2959 leader_election.cc:288] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 3 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:53.275372 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:53.275413 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb2742d409147deb30b47878472efbd" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:53.275549 2809 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.275580 2885 raft_consensus.cc:3032] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:12:53.275671 2885 raft_consensus.cc:729] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:53.275918 2885 consensus_queue.cc:260] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.276201 2885 raft_consensus.cc:3037] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.277741 2809 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 3.
I20240503 02:12:53.278187 2922 leader_election.cc:302] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:53.278445 2885 raft_consensus.cc:2454] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 3.
I20240503 02:12:53.278450 3130 raft_consensus.cc:2781] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:12:53.278872 3130 raft_consensus.cc:686] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 3 LEADER]: Becoming Leader. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:53.279181 3130 consensus_queue.cc:237] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.281507 2707 catalog_manager.cc:5484] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be reported cstate change: term changed from 2 to 3, leader changed from e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66) to 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67). New cstate: current_term: 3 leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: HEALTHY } } }
I20240503 02:12:53.413740 3132 raft_consensus.cc:979] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: : Instructing follower 0906dca094bd43a190401b5abde7bd21 to start an election
I20240503 02:12:53.413916 3132 raft_consensus.cc:1067] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Signalling peer 0906dca094bd43a190401b5abde7bd21 to start an election
I20240503 02:12:53.414629 2809 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "36a389246b014357973c5be29652951b"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
from {username='slave'} at 127.0.0.1:53794
I20240503 02:12:53.414839 2809 raft_consensus.cc:482] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:53.414995 2809 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.416989 2809 raft_consensus.cc:504] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.417804 2809 leader_election.cc:288] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 3 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:53.418224 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:53.418273 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a389246b014357973c5be29652951b" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:53.418428 2959 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.418452 2885 raft_consensus.cc:3032] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:12:53.418552 2885 raft_consensus.cc:729] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:53.418807 2885 consensus_queue.cc:260] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.419121 2885 raft_consensus.cc:3037] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.420711 2959 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 3.
I20240503 02:12:53.421165 2775 leader_election.cc:302] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:53.421262 2885 raft_consensus.cc:2454] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 3.
I20240503 02:12:53.421885 3139 raft_consensus.cc:2781] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:12:53.422305 3139 raft_consensus.cc:686] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 3 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:53.422586 3139 consensus_queue.cc:237] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.425182 2707 catalog_manager.cc:5484] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 2 to 3, leader changed from e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66) to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 3 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:53.570721 3131 raft_consensus.cc:979] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: : Instructing follower 0906dca094bd43a190401b5abde7bd21 to start an election
I20240503 02:12:53.570885 3131 raft_consensus.cc:1067] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Signalling peer 0906dca094bd43a190401b5abde7bd21 to start an election
I20240503 02:12:53.571471 2809 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "88c66cd2f65840209c29997708cee755"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
from {username='slave'} at 127.0.0.1:53794
I20240503 02:12:53.571692 2809 raft_consensus.cc:482] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:53.571797 2809 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.573673 2809 raft_consensus.cc:504] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.574189 2809 leader_election.cc:288] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 3 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:53.574617 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:53.574729 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88c66cd2f65840209c29997708cee755" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:53.574831 2885 raft_consensus.cc:3032] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:12:53.574893 2959 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.574954 2885 raft_consensus.cc:729] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:53.575256 2885 consensus_queue.cc:260] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.575626 2885 raft_consensus.cc:3037] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.578004 2959 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 3.
I20240503 02:12:53.578392 2775 leader_election.cc:302] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:53.578450 2885 raft_consensus.cc:2454] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 3.
I20240503 02:12:53.578668 3139 raft_consensus.cc:2781] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:12:53.578850 3139 raft_consensus.cc:686] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 3 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:53.579109 3139 consensus_queue.cc:237] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.581475 2707 catalog_manager.cc:5484] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 2 to 3, leader changed from e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66) to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 3 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:53.609611 3132 raft_consensus.cc:979] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: : Instructing follower 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:53.609795 3132 raft_consensus.cc:1067] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Signalling peer 1e80fe7edb59410e8c42c80d7bb325be to start an election
I20240503 02:12:53.610553 2959 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58"
dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
from {username='slave'} at 127.0.0.1:45938
I20240503 02:12:53.610836 2959 raft_consensus.cc:482] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:53.610997 2959 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.612995 2959 raft_consensus.cc:504] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:53.613581 2959 leader_election.cc:288] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 3 election: Requested vote from peers 0906dca094bd43a190401b5abde7bd21 (127.2.157.65:39937), e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167)
I20240503 02:12:53.614100 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:53.614122 2809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c77fed67fefe4d19a26f3aa1ce5faa58" candidate_uuid: "1e80fe7edb59410e8c42c80d7bb325be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0906dca094bd43a190401b5abde7bd21"
I20240503 02:12:53.614408 2885 raft_consensus.cc:3032] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:12:53.614409 2809 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.614677 2885 raft_consensus.cc:729] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:53.614984 2885 consensus_queue.cc:260] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:53.615427 2885 raft_consensus.cc:3037] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:12:53.617782 2809 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 3.
I20240503 02:12:53.618243 2922 leader_election.cc:302] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:53.618311 2885 raft_consensus.cc:2454] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80fe7edb59410e8c42c80d7bb325be in term 3.
I20240503 02:12:53.618642 3130 raft_consensus.cc:2781] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:12:53.618829 3130 raft_consensus.cc:686] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 LEADER]: Becoming Leader. State: Replica: 1e80fe7edb59410e8c42c80d7bb325be, State: Running, Role: LEADER
I20240503 02:12:53.619105 3130 consensus_queue.cc:237] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } }
I20240503 02:12:53.621829 2707 catalog_manager.cc:5484] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be reported cstate change: term changed from 2 to 3, leader changed from e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66) to 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67). New cstate: current_term: 3 leader_uuid: "1e80fe7edb59410e8c42c80d7bb325be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:53.702251 2809 raft_consensus.cc:1261] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:53.702818 3130 consensus_queue.cc:1035] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:53.707438 2885 raft_consensus.cc:1261] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:53.708196 3146 consensus_queue.cc:1035] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:53.817413 3131 raft_consensus.cc:979] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: : Instructing follower 0906dca094bd43a190401b5abde7bd21 to start an election
I20240503 02:12:53.817644 3132 raft_consensus.cc:1067] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Signalling peer 0906dca094bd43a190401b5abde7bd21 to start an election
I20240503 02:12:53.818279 2809 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43"
dest_uuid: "0906dca094bd43a190401b5abde7bd21"
from {username='slave'} at 127.0.0.1:53794
I20240503 02:12:53.818450 2809 raft_consensus.cc:482] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:12:53.818576 2809 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:12:53.820451 2809 raft_consensus.cc:504] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.821014 2809 leader_election.cc:288] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 4 election: Requested vote from peers e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66:39167), 1e80fe7edb59410e8c42c80d7bb325be (127.2.157.67:39391)
I20240503 02:12:53.821502 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e74960ef531445d9a285c8c8802ec5c6"
I20240503 02:12:53.821625 2959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b22bde32a4c41a7a60cf167ff1c4a43" candidate_uuid: "0906dca094bd43a190401b5abde7bd21" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1e80fe7edb59410e8c42c80d7bb325be"
I20240503 02:12:53.821743 2885 raft_consensus.cc:3032] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Stepping down as leader of term 3
I20240503 02:12:53.821821 2959 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:12:53.821851 2885 raft_consensus.cc:729] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e74960ef531445d9a285c8c8802ec5c6, State: Running, Role: LEADER
I20240503 02:12:53.822094 2885 consensus_queue.cc:260] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.822391 2885 raft_consensus.cc:3037] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:12:53.824028 2959 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 4.
I20240503 02:12:53.824283 2885 raft_consensus.cc:2454] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0906dca094bd43a190401b5abde7bd21 in term 4.
I20240503 02:12:53.824410 2775 leader_election.cc:302] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0906dca094bd43a190401b5abde7bd21, 1e80fe7edb59410e8c42c80d7bb325be; no voters:
I20240503 02:12:53.824700 3139 raft_consensus.cc:2781] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 02:12:53.824851 3139 raft_consensus.cc:686] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 4 LEADER]: Becoming Leader. State: Replica: 0906dca094bd43a190401b5abde7bd21, State: Running, Role: LEADER
I20240503 02:12:53.825111 3139 consensus_queue.cc:237] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } }
I20240503 02:12:53.827625 2705 catalog_manager.cc:5484] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 reported cstate change: term changed from 3 to 4, leader changed from e74960ef531445d9a285c8c8802ec5c6 (127.2.157.66) to 0906dca094bd43a190401b5abde7bd21 (127.2.157.65). New cstate: current_term: 4 leader_uuid: "0906dca094bd43a190401b5abde7bd21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20240503 02:12:53.980382 2885 raft_consensus.cc:1261] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Refusing update from remote peer 0906dca094bd43a190401b5abde7bd21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:53.981047 3139 consensus_queue.cc:1035] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:53.984807 2959 raft_consensus.cc:1261] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Refusing update from remote peer 0906dca094bd43a190401b5abde7bd21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:53.985569 3139 consensus_queue.cc:1035] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:54.054070 2809 raft_consensus.cc:1261] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:54.054615 3146 consensus_queue.cc:1035] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "0906dca094bd43a190401b5abde7bd21" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:54.056970 2959 raft_consensus.cc:1261] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Refusing update from remote peer 0906dca094bd43a190401b5abde7bd21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:54.058470 3139 consensus_queue.cc:1035] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:54.058988 2885 raft_consensus.cc:1261] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Refusing update from remote peer 1e80fe7edb59410e8c42c80d7bb325be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:54.059513 3133 consensus_queue.cc:1035] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:54.059665 2884 raft_consensus.cc:1261] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Refusing update from remote peer 0906dca094bd43a190401b5abde7bd21: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:12:54.060201 3139 consensus_queue.cc:1035] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:12:54.238957 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:54.240921 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:54.241114 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:54.270788 2885 raft_consensus.cc:1261] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 4 FOLLOWER]: Refusing update from remote peer 0906dca094bd43a190401b5abde7bd21: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 02:12:54.271368 3139 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e74960ef531445d9a285c8c8802ec5c6" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:12:54.275211 2959 raft_consensus.cc:1261] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 4 FOLLOWER]: Refusing update from remote peer 0906dca094bd43a190401b5abde7bd21: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 02:12:54.275826 3152 consensus_queue.cc:1035] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80fe7edb59410e8c42c80d7bb325be" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:12:55.241519 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:55.243778 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:55.243965 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:56.244345 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:56.245980 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:56.246224 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:57.246577 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:57.248183 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:57.248384 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:58.248735 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:58.250355 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:58.250557 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:12:59.250932 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:12:59.252820 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:12:59.252982 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:00.253412 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:00.255014 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:00.255213 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:01.255645 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:01.257750 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:01.257987 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:02.258350 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:02.260141 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:02.260329 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:03.281741 2677 auto_leader_rebalancer-test.cc:306] The leader distribution is
I20240503 02:13:03.282003 2677 tablet_server.cc:178] TabletServer@127.2.157.65:0 shutting down...
I20240503 02:13:03.294720 2677 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 02:13:03.295013 2677 tablet_replica.cc:330] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.295430 2677 raft_consensus.cc:2229] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.295701 2677 raft_consensus.cc:2258] T 5fb2742d409147deb30b47878472efbd P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.296850 2677 tablet_replica.cc:330] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.297015 2677 raft_consensus.cc:2229] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 4 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.297302 2677 raft_consensus.cc:2258] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 0906dca094bd43a190401b5abde7bd21 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.297785 2677 tablet_replica.cc:330] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.297927 2677 raft_consensus.cc:2229] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.298115 2677 raft_consensus.cc:2258] T c77fed67fefe4d19a26f3aa1ce5faa58 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.298534 2677 tablet_replica.cc:330] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.298688 2677 raft_consensus.cc:2229] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.298923 2677 raft_consensus.cc:2258] T 88c66cd2f65840209c29997708cee755 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.299355 2677 tablet_replica.cc:330] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.299496 2677 raft_consensus.cc:2229] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.299640 2677 raft_consensus.cc:2258] T 9f88e54e477d436184ce7cbed290e5c3 P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.300035 2677 tablet_replica.cc:330] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.300169 2677 raft_consensus.cc:2229] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.300307 2677 raft_consensus.cc:2258] T d68962f2fc7b43c19a544ab38811a430 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.300720 2677 tablet_replica.cc:330] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.300851 2677 raft_consensus.cc:2229] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.301082 2677 raft_consensus.cc:2258] T 36a389246b014357973c5be29652951b P 0906dca094bd43a190401b5abde7bd21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.301471 2677 tablet_replica.cc:330] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.301671 2677 raft_consensus.cc:2229] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.301821 2677 raft_consensus.cc:2258] T 148e7dde40e94ea3b153d3d168e47c2c P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.302215 2677 tablet_replica.cc:330] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21: stopping tablet replica
I20240503 02:13:03.302322 2677 raft_consensus.cc:2229] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.302482 2677 raft_consensus.cc:2258] T 41cb7656a4914a5b843598decbaa1fd0 P 0906dca094bd43a190401b5abde7bd21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.318991 2677 tablet_server.cc:194] TabletServer@127.2.157.65:0 shutdown complete.
I20240503 02:13:03.325575 2677 tablet_server.cc:178] TabletServer@127.2.157.66:0 shutting down...
I20240503 02:13:03.335784 2677 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 02:13:03.336232 2677 tablet_replica.cc:330] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.336403 2677 raft_consensus.cc:2229] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.336603 2677 raft_consensus.cc:2258] T 9b22bde32a4c41a7a60cf167ff1c4a43 P e74960ef531445d9a285c8c8802ec5c6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.337180 2677 tablet_replica.cc:330] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.337337 2677 raft_consensus.cc:2229] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.337682 2677 raft_consensus.cc:2258] T 9f88e54e477d436184ce7cbed290e5c3 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.338135 2677 tablet_replica.cc:330] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.338259 2677 raft_consensus.cc:2229] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.338388 2677 raft_consensus.cc:2258] T 5fb2742d409147deb30b47878472efbd P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.338850 2677 tablet_replica.cc:330] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.338990 2677 raft_consensus.cc:2229] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.339232 2677 raft_consensus.cc:2258] T d68962f2fc7b43c19a544ab38811a430 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.339658 2677 tablet_replica.cc:330] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.339782 2677 raft_consensus.cc:2229] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.339948 2677 raft_consensus.cc:2258] T 41cb7656a4914a5b843598decbaa1fd0 P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.340356 2677 tablet_replica.cc:330] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.340489 2677 raft_consensus.cc:2229] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.340610 2677 raft_consensus.cc:2258] T c77fed67fefe4d19a26f3aa1ce5faa58 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.341009 2677 tablet_replica.cc:330] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.341111 2677 raft_consensus.cc:2229] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.341246 2677 raft_consensus.cc:2258] T 88c66cd2f65840209c29997708cee755 P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.341655 2677 tablet_replica.cc:330] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.341771 2677 raft_consensus.cc:2229] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.341877 2677 raft_consensus.cc:2258] T 36a389246b014357973c5be29652951b P e74960ef531445d9a285c8c8802ec5c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.342232 2677 tablet_replica.cc:330] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6: stopping tablet replica
I20240503 02:13:03.342332 2677 raft_consensus.cc:2229] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.342555 2677 raft_consensus.cc:2258] T 148e7dde40e94ea3b153d3d168e47c2c P e74960ef531445d9a285c8c8802ec5c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.358863 2677 tablet_server.cc:194] TabletServer@127.2.157.66:0 shutdown complete.
I20240503 02:13:03.362831 2677 tablet_server.cc:178] TabletServer@127.2.157.67:0 shutting down...
I20240503 02:13:03.372502 2677 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 02:13:03.372753 2677 tablet_replica.cc:330] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.372936 2677 raft_consensus.cc:2229] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.373134 2677 raft_consensus.cc:2258] T 88c66cd2f65840209c29997708cee755 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.373728 2677 tablet_replica.cc:330] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.373889 2677 raft_consensus.cc:2229] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.374145 2677 raft_consensus.cc:2258] T 5fb2742d409147deb30b47878472efbd P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.374596 2677 tablet_replica.cc:330] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.374742 2677 raft_consensus.cc:2229] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.374897 2677 raft_consensus.cc:2258] T 9b22bde32a4c41a7a60cf167ff1c4a43 P 1e80fe7edb59410e8c42c80d7bb325be [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.375304 2677 tablet_replica.cc:330] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.375449 2677 raft_consensus.cc:2229] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.375581 2677 raft_consensus.cc:2258] T 9f88e54e477d436184ce7cbed290e5c3 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.376132 2677 tablet_replica.cc:330] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.376314 2677 raft_consensus.cc:2229] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.376602 2677 raft_consensus.cc:2258] T c77fed67fefe4d19a26f3aa1ce5faa58 P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.377075 2677 tablet_replica.cc:330] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.377250 2677 raft_consensus.cc:2229] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.377408 2677 raft_consensus.cc:2258] T 36a389246b014357973c5be29652951b P 1e80fe7edb59410e8c42c80d7bb325be [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.377903 2677 tablet_replica.cc:330] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.378085 2677 raft_consensus.cc:2229] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.378261 2677 raft_consensus.cc:2258] T 148e7dde40e94ea3b153d3d168e47c2c P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.378710 2677 tablet_replica.cc:330] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.378896 2677 raft_consensus.cc:2229] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:03.379055 2677 raft_consensus.cc:2258] T d68962f2fc7b43c19a544ab38811a430 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.379511 2677 tablet_replica.cc:330] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be: stopping tablet replica
I20240503 02:13:03.379658 2677 raft_consensus.cc:2229] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.379938 2677 raft_consensus.cc:2258] T 41cb7656a4914a5b843598decbaa1fd0 P 1e80fe7edb59410e8c42c80d7bb325be [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.395371 2677 tablet_server.cc:194] TabletServer@127.2.157.67:0 shutdown complete.
I20240503 02:13:03.399410 2677 master.cc:537] Master@127.2.157.126:34485 shutting down...
I20240503 02:13:03.407356 2677 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:13:03.407554 2677 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:03.407668 2677 tablet_replica.cc:330] T 00000000000000000000000000000000 P 69e51a9327804440a82be2eab3b07e93: stopping tablet replica
I20240503 02:13:03.420596 2677 master.cc:552] Master@127.2.157.126:34485 shutdown complete.
0906dca094bd43a190401b5abde7bd21 4
1e80fe7edb59410e8c42c80d7bb325be 1
e74960ef531445d9a285c8c8802ec5c6 4
0906dca094bd43a190401b5abde7bd21 3
1e80fe7edb59410e8c42c80d7bb325be 3
e74960ef531445d9a285c8c8802ec5c6 3
0906dca094bd43a190401b5abde7bd21 0
1e80fe7edb59410e8c42c80d7bb325be 1
e74960ef531445d9a285c8c8802ec5c6 8
0906dca094bd43a190401b5abde7bd21 3
1e80fe7edb59410e8c42c80d7bb325be 3
e74960ef531445d9a285c8c8802ec5c6 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (27776 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20240503 02:13:03.439193 2677 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.157.126:33835
I20240503 02:13:03.439769 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:03.442970 3166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:03.443704 3167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:03.444383 2677 server_base.cc:1034] running on GCE node
W20240503 02:13:03.444981 3169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:03.445349 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:03.445420 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:03.445469 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702383445442 us; error 0 us; skew 500 ppm
I20240503 02:13:03.445690 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:03.451435 2677 webserver.cc:460] Webserver started at http://127.2.157.126:46543/ using document root <none> and password file <none>
I20240503 02:13:03.451670 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:03.451764 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:03.451908 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:03.452551 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/master-0-root/instance:
uuid: "5a25ed62a3034c379554a1dfefa73cd2"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.455143 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:03.456743 3174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.457141 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20240503 02:13:03.457262 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/master-0-root
uuid: "5a25ed62a3034c379554a1dfefa73cd2"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.457396 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:03.465965 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:03.466481 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:03.480297 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.126:33835
I20240503 02:13:03.480331 3225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.126:33835 every 8 connection(s)
I20240503 02:13:03.481631 3226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.485675 3226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2: Bootstrap starting.
I20240503 02:13:03.486940 3226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.488502 3226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2: No bootstrap required, opened a new log
I20240503 02:13:03.489084 3226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25ed62a3034c379554a1dfefa73cd2" member_type: VOTER }
I20240503 02:13:03.489228 3226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.489289 3226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25ed62a3034c379554a1dfefa73cd2, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.489477 3226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25ed62a3034c379554a1dfefa73cd2" member_type: VOTER }
I20240503 02:13:03.489655 3226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:13:03.489733 3226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:13:03.489852 3226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:03.491477 3226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25ed62a3034c379554a1dfefa73cd2" member_type: VOTER }
I20240503 02:13:03.491683 3226 leader_election.cc:302] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a25ed62a3034c379554a1dfefa73cd2; no voters:
I20240503 02:13:03.492059 3226 leader_election.cc:288] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:13:03.492188 3229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:03.492700 3229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 5a25ed62a3034c379554a1dfefa73cd2, State: Running, Role: LEADER
I20240503 02:13:03.492980 3226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:13:03.492957 3229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25ed62a3034c379554a1dfefa73cd2" member_type: VOTER }
I20240503 02:13:03.494287 3230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a25ed62a3034c379554a1dfefa73cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25ed62a3034c379554a1dfefa73cd2" member_type: VOTER } }
I20240503 02:13:03.494361 3231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a25ed62a3034c379554a1dfefa73cd2. Latest consensus state: current_term: 1 leader_uuid: "5a25ed62a3034c379554a1dfefa73cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25ed62a3034c379554a1dfefa73cd2" member_type: VOTER } }
I20240503 02:13:03.494510 3230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [sys.catalog]: This master's current role is: LEADER
I20240503 02:13:03.494513 3231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [sys.catalog]: This master's current role is: LEADER
I20240503 02:13:03.495820 3235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:13:03.498672 3235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:13:03.499091 2677 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240503 02:13:03.501569 3235 catalog_manager.cc:1347] Generated new cluster ID: 39b8b54b12614b949efbdfa4bbb0395d
I20240503 02:13:03.501662 3235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:13:03.514293 3235 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:13:03.514962 3235 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:13:03.530189 3235 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2: Generated new TSK 0
I20240503 02:13:03.530387 3235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:13:03.563866 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:03.567533 3247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:03.568212 3248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:03.568845 2677 server_base.cc:1034] running on GCE node
W20240503 02:13:03.569384 3250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:03.569741 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:03.569844 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:03.569923 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702383569905 us; error 0 us; skew 500 ppm
I20240503 02:13:03.570091 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:03.571305 2677 webserver.cc:460] Webserver started at http://127.2.157.65:44509/ using document root <none> and password file <none>
I20240503 02:13:03.571529 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:03.571609 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:03.571755 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:03.572436 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-0-root/instance:
uuid: "2403b5a86c334221bd4caec2a6ad6451"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.574868 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20240503 02:13:03.576300 3255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.576601 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20240503 02:13:03.576707 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-0-root
uuid: "2403b5a86c334221bd4caec2a6ad6451"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.576836 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:03.598685 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:03.599305 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:03.599877 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:03.600731 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:03.600818 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.600921 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:03.601001 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.617165 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.65:43163
I20240503 02:13:03.617255 3317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.65:43163 every 8 connection(s)
I20240503 02:13:03.619015 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:03.622831 3322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:03.623567 3323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:03.624270 3325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:03.624735 2677 server_base.cc:1034] running on GCE node
I20240503 02:13:03.625236 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:03.625324 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:03.625411 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702383625357 us; error 0 us; skew 500 ppm
I20240503 02:13:03.625629 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:03.626842 3318 heartbeater.cc:347] Connected to a master server at 127.2.157.126:33835
I20240503 02:13:03.627009 3318 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:03.627007 2677 webserver.cc:460] Webserver started at http://127.2.157.66:36981/ using document root <none> and password file <none>
I20240503 02:13:03.627427 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:03.627434 3318 heartbeater.cc:511] Master 127.2.157.126:33835 requested a full tablet report, sending...
I20240503 02:13:03.627568 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:03.627712 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:03.628312 3191 ts_manager.cc:194] Registered new tserver with Master: 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163)
I20240503 02:13:03.628419 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-1-root/instance:
uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.629371 3191 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51344
I20240503 02:13:03.630992 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20240503 02:13:03.632477 3330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.632815 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20240503 02:13:03.632943 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-1-root
uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.633085 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:03.644012 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:03.644694 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:03.645303 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:03.646381 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:03.646483 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.646603 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:03.646659 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.660962 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.66:42373
I20240503 02:13:03.660981 3392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.66:42373 every 8 connection(s)
I20240503 02:13:03.662662 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:03.666312 3396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:03.667078 3397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:03.667754 3399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:03.668365 2677 server_base.cc:1034] running on GCE node
I20240503 02:13:03.668687 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:03.668831 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:03.668964 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702383668929 us; error 0 us; skew 500 ppm
I20240503 02:13:03.669173 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:03.669771 3393 heartbeater.cc:347] Connected to a master server at 127.2.157.126:33835
I20240503 02:13:03.669906 3393 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:03.670253 3393 heartbeater.cc:511] Master 127.2.157.126:33835 requested a full tablet report, sending...
I20240503 02:13:03.671123 3191 ts_manager.cc:194] Registered new tserver with Master: bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:03.671228 2677 webserver.cc:460] Webserver started at http://127.2.157.67:41719/ using document root <none> and password file <none>
I20240503 02:13:03.671514 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:03.671628 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:03.671789 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:03.671972 3191 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51358
I20240503 02:13:03.672658 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-2-root/instance:
uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.675262 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:03.677151 3404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.677623 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.001s
I20240503 02:13:03.677748 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-2-root
uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
format_stamp: "Formatted at 2024-05-03 02:13:03 on dist-test-slave-tz5s"
I20240503 02:13:03.677872 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1714702355592969-2677-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:03.697233 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:03.697824 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:03.698392 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:03.699163 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:03.699265 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.699365 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:03.699445 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:03.715579 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.67:33707
I20240503 02:13:03.715631 3466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.67:33707 every 8 connection(s)
I20240503 02:13:03.722692 3467 heartbeater.cc:347] Connected to a master server at 127.2.157.126:33835
I20240503 02:13:03.722820 3467 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:03.723181 3467 heartbeater.cc:511] Master 127.2.157.126:33835 requested a full tablet report, sending...
I20240503 02:13:03.723997 3191 ts_manager.cc:194] Registered new tserver with Master: a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:03.724573 2677 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007959821s
I20240503 02:13:03.724810 3191 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51370
I20240503 02:13:03.725838 2677 test_util.cc:268] Using random seed: 1115307246
I20240503 02:13:03.737659 3191 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51386:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:13:03.738467 3191 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:13:03.773969 3356 tablet_service.cc:1467] Processing CreateTablet for tablet d451172292d243818f33c121270e4888 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240503 02:13:03.774052 3355 tablet_service.cc:1467] Processing CreateTablet for tablet 39eeea4f75de433eb3e1f744b3f4e31a (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240503 02:13:03.774479 3351 tablet_service.cc:1467] Processing CreateTablet for tablet e356847e2c24487aaa5a99f3071efecd (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240503 02:13:03.774821 3353 tablet_service.cc:1467] Processing CreateTablet for tablet b9040d9c44fa4791a3f5e17dcb896f32 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240503 02:13:03.773415 3357 tablet_service.cc:1467] Processing CreateTablet for tablet 8b5f97b3837f454d824325797e30ae76 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240503 02:13:03.775161 3352 tablet_service.cc:1467] Processing CreateTablet for tablet 837ab97c37244c3384d348e4b391b132 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240503 02:13:03.775235 3350 tablet_service.cc:1467] Processing CreateTablet for tablet b5abf63247bb47c294c1e7c45f6d26a0 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240503 02:13:03.775442 3354 tablet_service.cc:1467] Processing CreateTablet for tablet bc754ea99c59493abae00c50d74c3e02 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240503 02:13:03.775477 3349 tablet_service.cc:1467] Processing CreateTablet for tablet 2ac4be8db7d546d3a8a75f14203d95ab (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240503 02:13:03.773370 3358 tablet_service.cc:1467] Processing CreateTablet for tablet 8d488e03df854d38830d4f004e7d3a4d (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240503 02:13:03.777366 3356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d451172292d243818f33c121270e4888. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.777846 3350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5abf63247bb47c294c1e7c45f6d26a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.779438 3354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc754ea99c59493abae00c50d74c3e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.779913 3281 tablet_service.cc:1467] Processing CreateTablet for tablet d451172292d243818f33c121270e4888 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240503 02:13:03.780510 3281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d451172292d243818f33c121270e4888. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.780654 3282 tablet_service.cc:1467] Processing CreateTablet for tablet 8b5f97b3837f454d824325797e30ae76 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240503 02:13:03.781142 3282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b5f97b3837f454d824325797e30ae76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.781944 3283 tablet_service.cc:1467] Processing CreateTablet for tablet 8d488e03df854d38830d4f004e7d3a4d (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240503 02:13:03.786062 3276 tablet_service.cc:1467] Processing CreateTablet for tablet e356847e2c24487aaa5a99f3071efecd (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240503 02:13:03.786391 3355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39eeea4f75de433eb3e1f744b3f4e31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.786574 3283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d488e03df854d38830d4f004e7d3a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.786501 3279 tablet_service.cc:1467] Processing CreateTablet for tablet bc754ea99c59493abae00c50d74c3e02 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240503 02:13:03.790155 3349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ac4be8db7d546d3a8a75f14203d95ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.791600 3357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b5f97b3837f454d824325797e30ae76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.791707 3276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e356847e2c24487aaa5a99f3071efecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.790149 3278 tablet_service.cc:1467] Processing CreateTablet for tablet b9040d9c44fa4791a3f5e17dcb896f32 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240503 02:13:03.789189 3280 tablet_service.cc:1467] Processing CreateTablet for tablet 39eeea4f75de433eb3e1f744b3f4e31a (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240503 02:13:03.795240 3279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc754ea99c59493abae00c50d74c3e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.795639 3358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d488e03df854d38830d4f004e7d3a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.792146 3275 tablet_service.cc:1467] Processing CreateTablet for tablet b5abf63247bb47c294c1e7c45f6d26a0 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240503 02:13:03.795213 3274 tablet_service.cc:1467] Processing CreateTablet for tablet 2ac4be8db7d546d3a8a75f14203d95ab (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240503 02:13:03.795279 3432 tablet_service.cc:1467] Processing CreateTablet for tablet 8d488e03df854d38830d4f004e7d3a4d (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20240503 02:13:03.797760 3277 tablet_service.cc:1467] Processing CreateTablet for tablet 837ab97c37244c3384d348e4b391b132 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240503 02:13:03.799289 3352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837ab97c37244c3384d348e4b391b132. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.799670 3351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e356847e2c24487aaa5a99f3071efecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.800004 3278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9040d9c44fa4791a3f5e17dcb896f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.801256 3426 tablet_service.cc:1467] Processing CreateTablet for tablet 837ab97c37244c3384d348e4b391b132 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20240503 02:13:03.800505 3431 tablet_service.cc:1467] Processing CreateTablet for tablet 8b5f97b3837f454d824325797e30ae76 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20240503 02:13:03.802668 3353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9040d9c44fa4791a3f5e17dcb896f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.802201 3427 tablet_service.cc:1467] Processing CreateTablet for tablet b9040d9c44fa4791a3f5e17dcb896f32 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20240503 02:13:03.802289 3429 tablet_service.cc:1467] Processing CreateTablet for tablet 39eeea4f75de433eb3e1f744b3f4e31a (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20240503 02:13:03.803442 3425 tablet_service.cc:1467] Processing CreateTablet for tablet e356847e2c24487aaa5a99f3071efecd (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20240503 02:13:03.803758 3280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39eeea4f75de433eb3e1f744b3f4e31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.807068 3277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837ab97c37244c3384d348e4b391b132. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.802430 3428 tablet_service.cc:1467] Processing CreateTablet for tablet bc754ea99c59493abae00c50d74c3e02 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20240503 02:13:03.803269 3430 tablet_service.cc:1467] Processing CreateTablet for tablet d451172292d243818f33c121270e4888 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20240503 02:13:03.803295 3424 tablet_service.cc:1467] Processing CreateTablet for tablet b5abf63247bb47c294c1e7c45f6d26a0 (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20240503 02:13:03.807135 3423 tablet_service.cc:1467] Processing CreateTablet for tablet 2ac4be8db7d546d3a8a75f14203d95ab (DEFAULT_TABLE table=test-workload [id=33c97b8d469d40c3b11e615e9594b13d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20240503 02:13:03.810647 3432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d488e03df854d38830d4f004e7d3a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.810894 3275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5abf63247bb47c294c1e7c45f6d26a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.812592 3274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ac4be8db7d546d3a8a75f14203d95ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.815635 3427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9040d9c44fa4791a3f5e17dcb896f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.817771 3429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39eeea4f75de433eb3e1f744b3f4e31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.818017 3423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ac4be8db7d546d3a8a75f14203d95ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.818222 3424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5abf63247bb47c294c1e7c45f6d26a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.818437 3431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b5f97b3837f454d824325797e30ae76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.821058 3425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e356847e2c24487aaa5a99f3071efecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.823359 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d451172292d243818f33c121270e4888. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.826386 3426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837ab97c37244c3384d348e4b391b132. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.827270 3428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc754ea99c59493abae00c50d74c3e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:03.889183 3482 tablet_bootstrap.cc:492] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.898945 3483 tablet_bootstrap.cc:492] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:03.913144 3484 tablet_bootstrap.cc:492] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:03.916967 3482 tablet_bootstrap.cc:654] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.917917 3483 tablet_bootstrap.cc:654] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.919912 3484 tablet_bootstrap.cc:654] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.920534 3483 tablet_bootstrap.cc:492] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:03.920658 3483 ts_tablet_manager.cc:1367] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.000s
I20240503 02:13:03.921180 3482 tablet_bootstrap.cc:492] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:03.921314 3482 ts_tablet_manager.cc:1367] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.005s
I20240503 02:13:03.921273 3483 raft_consensus.cc:348] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.921480 3483 raft_consensus.cc:374] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.921609 3483 raft_consensus.cc:729] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.921873 3483 consensus_queue.cc:260] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.921998 3482 raft_consensus.cc:348] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.922180 3482 raft_consensus.cc:374] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.922279 3482 raft_consensus.cc:729] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.922559 3482 consensus_queue.cc:260] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.922734 3318 heartbeater.cc:503] Master 127.2.157.126:33835 was elected leader, sending a full tablet report...
I20240503 02:13:03.923398 3393 heartbeater.cc:503] Master 127.2.157.126:33835 was elected leader, sending a full tablet report...
I20240503 02:13:03.923418 3482 ts_tablet_manager.cc:1398] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:03.923941 3482 tablet_bootstrap.cc:492] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.925825 3482 tablet_bootstrap.cc:654] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.925827 3484 tablet_bootstrap.cc:492] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:03.926056 3484 ts_tablet_manager.cc:1367] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20240503 02:13:03.926860 3484 raft_consensus.cc:348] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.927121 3484 raft_consensus.cc:374] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.927227 3484 raft_consensus.cc:729] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.927460 3484 consensus_queue.cc:260] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.928225 3482 tablet_bootstrap.cc:492] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:03.928269 3483 ts_tablet_manager.cc:1398] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20240503 02:13:03.928555 3482 ts_tablet_manager.cc:1367] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20240503 02:13:03.929625 3482 raft_consensus.cc:348] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.929982 3482 raft_consensus.cc:374] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.930155 3482 raft_consensus.cc:729] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.930039 3483 tablet_bootstrap.cc:492] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:03.930564 3482 consensus_queue.cc:260] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.931355 3482 ts_tablet_manager.cc:1398] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240503 02:13:03.931738 3482 tablet_bootstrap.cc:492] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.933465 3483 tablet_bootstrap.cc:654] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.934080 3482 tablet_bootstrap.cc:654] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.934854 3467 heartbeater.cc:503] Master 127.2.157.126:33835 was elected leader, sending a full tablet report...
I20240503 02:13:03.934888 3484 ts_tablet_manager.cc:1398] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20240503 02:13:03.935295 3484 tablet_bootstrap.cc:492] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:03.937333 3484 tablet_bootstrap.cc:654] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.941789 3483 tablet_bootstrap.cc:492] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:03.941957 3483 ts_tablet_manager.cc:1367] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20240503 02:13:03.942726 3483 raft_consensus.cc:348] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.942909 3483 raft_consensus.cc:374] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.942975 3483 raft_consensus.cc:729] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.942977 3484 tablet_bootstrap.cc:492] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:03.943176 3484 ts_tablet_manager.cc:1367] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20240503 02:13:03.943257 3483 consensus_queue.cc:260] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.943392 3482 tablet_bootstrap.cc:492] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:03.943558 3482 ts_tablet_manager.cc:1367] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20240503 02:13:03.943897 3484 raft_consensus.cc:348] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.944120 3484 raft_consensus.cc:374] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.944217 3484 raft_consensus.cc:729] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.944332 3482 raft_consensus.cc:348] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.944540 3482 raft_consensus.cc:374] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.944451 3484 consensus_queue.cc:260] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.944684 3482 raft_consensus.cc:729] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.944926 3482 consensus_queue.cc:260] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.945068 3484 ts_tablet_manager.cc:1398] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:03.945386 3484 tablet_bootstrap.cc:492] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:03.946086 3482 ts_tablet_manager.cc:1398] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:13:03.946403 3482 tablet_bootstrap.cc:492] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.947057 3483 ts_tablet_manager.cc:1398] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240503 02:13:03.947057 3484 tablet_bootstrap.cc:654] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.948112 3482 tablet_bootstrap.cc:654] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.948181 3483 tablet_bootstrap.cc:492] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:03.950351 3483 tablet_bootstrap.cc:654] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.952963 3483 tablet_bootstrap.cc:492] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:03.953112 3483 ts_tablet_manager.cc:1367] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:13:03.953471 3484 tablet_bootstrap.cc:492] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:03.953635 3484 ts_tablet_manager.cc:1367] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.001s
I20240503 02:13:03.953899 3483 raft_consensus.cc:348] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.954094 3483 raft_consensus.cc:374] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.954195 3483 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.954367 3484 raft_consensus.cc:348] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.954454 3483 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.954555 3484 raft_consensus.cc:374] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.954641 3484 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.954860 3484 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.955369 3484 ts_tablet_manager.cc:1398] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:03.955813 3483 ts_tablet_manager.cc:1398] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:03.956100 3483 tablet_bootstrap.cc:492] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:03.956228 3484 tablet_bootstrap.cc:492] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:03.957108 3482 tablet_bootstrap.cc:492] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:03.957285 3482 ts_tablet_manager.cc:1367] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20240503 02:13:03.957906 3484 tablet_bootstrap.cc:654] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.957943 3483 tablet_bootstrap.cc:654] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.958171 3482 raft_consensus.cc:348] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.958408 3482 raft_consensus.cc:374] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.958498 3482 raft_consensus.cc:729] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.958794 3482 consensus_queue.cc:260] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.960932 3482 ts_tablet_manager.cc:1398] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20240503 02:13:03.961225 3482 tablet_bootstrap.cc:492] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.962796 3482 tablet_bootstrap.cc:654] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.964630 3482 tablet_bootstrap.cc:492] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:03.964779 3482 ts_tablet_manager.cc:1367] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240503 02:13:03.965307 3482 raft_consensus.cc:348] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.965456 3482 raft_consensus.cc:374] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.965576 3482 raft_consensus.cc:729] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.965773 3482 consensus_queue.cc:260] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.966209 3482 ts_tablet_manager.cc:1398] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240503 02:13:03.966457 3482 tablet_bootstrap.cc:492] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.968092 3482 tablet_bootstrap.cc:654] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.969229 3489 raft_consensus.cc:482] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:03.969381 3489 raft_consensus.cc:504] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.970067 3484 tablet_bootstrap.cc:492] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:03.970229 3484 ts_tablet_manager.cc:1367] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.001s
I20240503 02:13:03.970274 3489 leader_election.cc:288] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:03.970870 3483 tablet_bootstrap.cc:492] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:03.971014 3483 ts_tablet_manager.cc:1367] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.003s
I20240503 02:13:03.970948 3484 raft_consensus.cc:348] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.971132 3484 raft_consensus.cc:374] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.971218 3484 raft_consensus.cc:729] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.971472 3484 consensus_queue.cc:260] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.971683 3483 raft_consensus.cc:348] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.971868 3483 raft_consensus.cc:374] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.971942 3483 raft_consensus.cc:729] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.972079 3484 ts_tablet_manager.cc:1398] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:03.972360 3484 tablet_bootstrap.cc:492] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:03.972231 3483 consensus_queue.cc:260] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.973495 3483 ts_tablet_manager.cc:1398] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:03.973827 3483 tablet_bootstrap.cc:492] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:03.974079 3484 tablet_bootstrap.cc:654] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.975623 3483 tablet_bootstrap.cc:654] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.979847 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
W20240503 02:13:03.981516 3332 leader_election.cc:341] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:03.990227 3482 tablet_bootstrap.cc:492] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:03.990545 3482 ts_tablet_manager.cc:1367] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.003s
I20240503 02:13:03.991647 3482 raft_consensus.cc:348] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.992023 3482 raft_consensus.cc:374] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:03.992192 3482 raft_consensus.cc:729] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:03.992560 3482 consensus_queue.cc:260] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:03.993467 3482 ts_tablet_manager.cc:1398] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240503 02:13:03.993948 3482 tablet_bootstrap.cc:492] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:03.996425 3482 tablet_bootstrap.cc:654] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:03.999788 3484 tablet_bootstrap.cc:492] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:03.999945 3484 ts_tablet_manager.cc:1367] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.028s user 0.017s sys 0.009s
I20240503 02:13:04.000317 3483 tablet_bootstrap.cc:492] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:04.000461 3483 ts_tablet_manager.cc:1367] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.011s
I20240503 02:13:04.000684 3482 tablet_bootstrap.cc:492] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:04.000725 3484 raft_consensus.cc:348] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.000939 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.001070 3483 raft_consensus.cc:348] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.001170 3482 ts_tablet_manager.cc:1367] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20240503 02:13:04.001248 3483 raft_consensus.cc:374] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.001381 3483 raft_consensus.cc:729] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.001570 3484 raft_consensus.cc:374] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 02:13:04.001713 3331 leader_election.cc:341] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:04.001754 3484 raft_consensus.cc:729] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.001659 3483 consensus_queue.cc:260] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.001917 3331 leader_election.cc:302] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcb3dc993fb340d19a30629ac22f6b0c; no voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04
I20240503 02:13:04.001971 3482 raft_consensus.cc:348] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.002168 3482 raft_consensus.cc:374] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.002143 3484 consensus_queue.cc:260] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.002269 3489 raft_consensus.cc:2726] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:04.002274 3482 raft_consensus.cc:729] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.002488 3482 consensus_queue.cc:260] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.002957 3482 ts_tablet_manager.cc:1398] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:04.003226 3484 ts_tablet_manager.cc:1398] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240503 02:13:04.003233 3482 tablet_bootstrap.cc:492] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:04.003513 3484 tablet_bootstrap.cc:492] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:04.004370 3483 ts_tablet_manager.cc:1398] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20240503 02:13:04.005090 3482 tablet_bootstrap.cc:654] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.005293 3484 tablet_bootstrap.cc:654] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.007136 3483 tablet_bootstrap.cc:492] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:04.009434 3483 tablet_bootstrap.cc:654] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.013150 3482 tablet_bootstrap.cc:492] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:04.013274 3482 ts_tablet_manager.cc:1367] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20240503 02:13:04.013670 3484 tablet_bootstrap.cc:492] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:04.013873 3484 ts_tablet_manager.cc:1367] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.010s user 0.010s sys 0.000s
I20240503 02:13:04.014050 3482 raft_consensus.cc:348] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.014236 3482 raft_consensus.cc:374] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.014297 3482 raft_consensus.cc:729] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.014499 3482 consensus_queue.cc:260] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.014629 3484 raft_consensus.cc:348] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.014848 3483 tablet_bootstrap.cc:492] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:04.014848 3484 raft_consensus.cc:374] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.014961 3482 ts_tablet_manager.cc:1398] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:04.015081 3484 raft_consensus.cc:729] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.015219 3483 ts_tablet_manager.cc:1367] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20240503 02:13:04.015343 3491 raft_consensus.cc:482] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.015497 3491 raft_consensus.cc:504] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.015542 3484 consensus_queue.cc:260] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.016265 3483 raft_consensus.cc:348] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.016566 3491 leader_election.cc:288] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:04.016628 3483 raft_consensus.cc:374] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.016855 3483 raft_consensus.cc:729] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.017298 3483 consensus_queue.cc:260] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.019310 3483 ts_tablet_manager.cc:1398] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:04.019743 3483 tablet_bootstrap.cc:492] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:04.025666 3482 tablet_bootstrap.cc:492] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:04.027118 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.027392 3293 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 0.
I20240503 02:13:04.027992 3406 leader_election.cc:302] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.028465 3491 raft_consensus.cc:2781] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.028604 3491 raft_consensus.cc:482] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.028715 3491 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.029207 3482 tablet_bootstrap.cc:654] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.032840 3491 raft_consensus.cc:504] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.033963 3482 tablet_bootstrap.cc:492] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:04.034706 3482 ts_tablet_manager.cc:1367] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.004s
I20240503 02:13:04.035398 3482 raft_consensus.cc:348] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.035619 3482 raft_consensus.cc:374] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.035718 3482 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.035660 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.035912 3293 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.035976 3482 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.038769 3482 ts_tablet_manager.cc:1398] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:04.039853 3293 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 1.
I20240503 02:13:04.039898 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" is_pre_election: true
I20240503 02:13:04.040232 3368 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 0.
I20240503 02:13:04.040351 3406 leader_election.cc:302] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.040585 3491 leader_election.cc:288] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:04.040823 3482 tablet_bootstrap.cc:492] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: Bootstrap starting.
I20240503 02:13:04.040830 3491 raft_consensus.cc:2781] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.040997 3491 raft_consensus.cc:686] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:04.041182 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.041359 3368 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.041278 3491 consensus_queue.cc:237] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.022001 3483 tablet_bootstrap.cc:654] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.042680 3482 tablet_bootstrap.cc:654] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.044584 3368 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 1.
I20240503 02:13:04.046229 3484 ts_tablet_manager.cc:1398] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.032s user 0.012s sys 0.017s
I20240503 02:13:04.046659 3484 tablet_bootstrap.cc:492] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:04.047590 3483 tablet_bootstrap.cc:492] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:04.047912 3483 ts_tablet_manager.cc:1367] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.000s
I20240503 02:13:04.048822 3483 raft_consensus.cc:348] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.049114 3483 raft_consensus.cc:374] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.049379 3483 raft_consensus.cc:729] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.049496 3488 raft_consensus.cc:482] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.049124 3484 tablet_bootstrap.cc:654] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.049688 3488 raft_consensus.cc:504] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.049762 3190 catalog_manager.cc:5484] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 0 to 1, leader changed from <none> to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 1 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:04.050238 3483 consensus_queue.cc:260] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.051133 3488 leader_election.cc:288] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.054797 3482 tablet_bootstrap.cc:492] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: No bootstrap required, opened a new log
I20240503 02:13:04.055007 3482 ts_tablet_manager.cc:1367] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20240503 02:13:04.055980 3482 raft_consensus.cc:348] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.056291 3482 raft_consensus.cc:374] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.056382 3482 raft_consensus.cc:729] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.056711 3482 consensus_queue.cc:260] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.060530 3482 ts_tablet_manager.cc:1398] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:13:04.070804 3483 ts_tablet_manager.cc:1398] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.023s user 0.004s sys 0.001s
I20240503 02:13:04.070910 3484 tablet_bootstrap.cc:492] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:04.071094 3484 ts_tablet_manager.cc:1367] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.013s
I20240503 02:13:04.071205 3483 tablet_bootstrap.cc:492] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:04.071715 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.071903 3484 raft_consensus.cc:348] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.071967 3442 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 0.
I20240503 02:13:04.072256 3484 raft_consensus.cc:374] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.072378 3484 raft_consensus.cc:729] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.072559 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" is_pre_election: true
I20240503 02:13:04.072758 3368 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 0.
I20240503 02:13:04.072676 3484 consensus_queue.cc:260] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.073180 3483 tablet_bootstrap.cc:654] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.073387 3484 ts_tablet_manager.cc:1398] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240503 02:13:04.073747 3256 leader_election.cc:302] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.074359 3515 raft_consensus.cc:2781] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.074479 3515 raft_consensus.cc:482] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.074700 3484 tablet_bootstrap.cc:492] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:04.075404 3515 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.076172 3483 tablet_bootstrap.cc:492] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:04.076380 3483 ts_tablet_manager.cc:1367] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20240503 02:13:04.076422 3484 tablet_bootstrap.cc:654] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.077270 3483 raft_consensus.cc:348] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.077704 3483 raft_consensus.cc:374] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.077827 3483 raft_consensus.cc:729] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.078122 3483 consensus_queue.cc:260] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.078660 3484 tablet_bootstrap.cc:492] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:04.078706 3483 ts_tablet_manager.cc:1398] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:04.078837 3484 ts_tablet_manager.cc:1367] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240503 02:13:04.079074 3483 tablet_bootstrap.cc:492] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:04.079640 3484 raft_consensus.cc:348] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.079854 3484 raft_consensus.cc:374] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.079955 3484 raft_consensus.cc:729] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.080202 3484 consensus_queue.cc:260] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.080956 3483 tablet_bootstrap.cc:654] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.081336 3484 ts_tablet_manager.cc:1398] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:04.081694 3484 tablet_bootstrap.cc:492] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:04.083107 3483 tablet_bootstrap.cc:492] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:04.083312 3483 ts_tablet_manager.cc:1367] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:13:04.083400 3484 tablet_bootstrap.cc:654] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.084300 3483 raft_consensus.cc:348] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.084666 3483 raft_consensus.cc:374] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.084849 3483 raft_consensus.cc:729] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.085206 3483 consensus_queue.cc:260] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.085815 3483 ts_tablet_manager.cc:1398] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:04.086122 3484 tablet_bootstrap.cc:492] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:04.086210 3483 tablet_bootstrap.cc:492] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451: Bootstrap starting.
I20240503 02:13:04.086355 3484 ts_tablet_manager.cc:1367] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240503 02:13:04.087121 3515 raft_consensus.cc:504] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.087183 3484 raft_consensus.cc:348] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.087524 3484 raft_consensus.cc:374] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.087672 3484 raft_consensus.cc:729] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.087954 3483 tablet_bootstrap.cc:654] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.088078 3484 consensus_queue.cc:260] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.088264 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.088503 3368 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.088744 3484 ts_tablet_manager.cc:1398] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:04.089095 3484 tablet_bootstrap.cc:492] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04: Bootstrap starting.
I20240503 02:13:04.090766 3368 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 1.
I20240503 02:13:04.091503 3484 tablet_bootstrap.cc:654] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:04.091495 3256 leader_election.cc:302] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.092408 3488 raft_consensus.cc:2781] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.094765 3488 raft_consensus.cc:686] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.095074 3488 consensus_queue.cc:237] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.097816 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.098074 3442 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.098129 3189 catalog_manager.cc:5484] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.100631 3442 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 1.
I20240503 02:13:04.101119 3515 leader_election.cc:288] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.102703 3484 tablet_bootstrap.cc:492] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04: No bootstrap required, opened a new log
I20240503 02:13:04.102705 3483 tablet_bootstrap.cc:492] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451: No bootstrap required, opened a new log
I20240503 02:13:04.102927 3483 ts_tablet_manager.cc:1367] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.003s
I20240503 02:13:04.102972 3484 ts_tablet_manager.cc:1367] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20240503 02:13:04.103813 3483 raft_consensus.cc:348] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.103811 3484 raft_consensus.cc:348] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.104097 3483 raft_consensus.cc:374] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.104123 3484 raft_consensus.cc:374] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:04.104207 3483 raft_consensus.cc:729] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.104216 3484 raft_consensus.cc:729] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Initialized, Role: FOLLOWER
I20240503 02:13:04.104503 3483 consensus_queue.cc:260] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.104480 3484 consensus_queue.cc:260] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.105062 3483 ts_tablet_manager.cc:1398] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:04.105063 3484 ts_tablet_manager.cc:1398] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:04.110090 3489 raft_consensus.cc:482] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.110211 3489 raft_consensus.cc:504] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.110740 3489 leader_election.cc:288] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.111060 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837ab97c37244c3384d348e4b391b132" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.111142 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837ab97c37244c3384d348e4b391b132" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.111301 3442 raft_consensus.cc:2454] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.111299 3293 raft_consensus.cc:2454] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.111691 3331 leader_election.cc:302] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.111920 3489 raft_consensus.cc:2781] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.112015 3489 raft_consensus.cc:482] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.112113 3489 raft_consensus.cc:3037] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.113927 3489 raft_consensus.cc:504] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.114375 3489 leader_election.cc:288] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.114723 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837ab97c37244c3384d348e4b391b132" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.114809 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837ab97c37244c3384d348e4b391b132" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.114883 3293 raft_consensus.cc:3037] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.114995 3442 raft_consensus.cc:3037] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.116873 3293 raft_consensus.cc:2454] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.116926 3442 raft_consensus.cc:2454] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.117234 3331 leader_election.cc:302] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.117548 3489 raft_consensus.cc:2781] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.117939 3489 raft_consensus.cc:686] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.118217 3489 consensus_queue.cc:237] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.120796 3190 catalog_manager.cc:5484] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 1 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.140316 3515 raft_consensus.cc:482] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.140524 3515 raft_consensus.cc:504] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.141036 3515 leader_election.cc:288] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.141506 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" is_pre_election: true
I20240503 02:13:04.141645 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.141726 3368 raft_consensus.cc:2454] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 0.
I20240503 02:13:04.141899 3442 raft_consensus.cc:2454] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 0.
I20240503 02:13:04.142109 3256 leader_election.cc:302] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.142364 3515 raft_consensus.cc:2781] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.142448 3515 raft_consensus.cc:482] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.142546 3515 raft_consensus.cc:3037] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.144327 3489 raft_consensus.cc:482] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.144331 3526 raft_consensus.cc:482] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.144295 3515 raft_consensus.cc:504] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.144508 3489 raft_consensus.cc:504] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.144515 3526 raft_consensus.cc:504] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.145001 3515 leader_election.cc:288] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.145251 3489 leader_election.cc:288] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.145426 3526 leader_election.cc:288] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.145833 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.146023 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.145916 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.146250 3442 raft_consensus.cc:2454] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.146186 3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.146344 3441 raft_consensus.cc:3037] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.146247 3292 raft_consensus.cc:2454] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.146587 3440 raft_consensus.cc:2454] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.146652 3331 leader_election.cc:302] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.146768 3332 leader_election.cc:302] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.146952 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.147006 3526 raft_consensus.cc:2781] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.147178 3368 raft_consensus.cc:3037] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.147236 3526 raft_consensus.cc:482] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.147361 3526 raft_consensus.cc:3037] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.145840 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.148310 3293 raft_consensus.cc:2454] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.148294 3489 raft_consensus.cc:2781] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.148620 3489 raft_consensus.cc:482] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.148785 3489 raft_consensus.cc:3037] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.150198 3441 raft_consensus.cc:2454] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 1.
I20240503 02:13:04.150804 3368 raft_consensus.cc:2454] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 1.
I20240503 02:13:04.150883 3256 leader_election.cc:302] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.150897 3526 raft_consensus.cc:504] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.151361 3515 raft_consensus.cc:2781] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.151508 3526 leader_election.cc:288] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.151571 3515 raft_consensus.cc:686] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.151942 3442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.151851 3515 consensus_queue.cc:237] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.151965 3489 raft_consensus.cc:504] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.152128 3442 raft_consensus.cc:3037] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.152606 3489 leader_election.cc:288] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.153087 3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.153255 3440 raft_consensus.cc:3037] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.153687 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.153898 3293 raft_consensus.cc:3037] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.155123 3442 raft_consensus.cc:2454] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.155970 3293 raft_consensus.cc:2454] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.155972 3440 raft_consensus.cc:2454] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.156363 3332 leader_election.cc:302] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.156785 3489 raft_consensus.cc:2781] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.156949 3489 raft_consensus.cc:686] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.157049 3331 leader_election.cc:302] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.156359 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.157222 3489 consensus_queue.cc:237] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.157353 3526 raft_consensus.cc:2781] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.157500 3526 raft_consensus.cc:686] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.157899 3526 consensus_queue.cc:237] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.158717 3190 catalog_manager.cc:5484] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.157354 3292 raft_consensus.cc:3037] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.164021 3292 raft_consensus.cc:2454] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.166901 3190 catalog_manager.cc:5484] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 1 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.170629 3190 catalog_manager.cc:5484] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 1 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.189321 3526 raft_consensus.cc:482] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.189567 3526 raft_consensus.cc:504] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.190151 3526 leader_election.cc:288] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.190645 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.190654 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.190917 3441 raft_consensus.cc:2454] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.190950 3292 raft_consensus.cc:2454] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.191311 3331 leader_election.cc:302] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.191576 3526 raft_consensus.cc:2781] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.191669 3526 raft_consensus.cc:482] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.191776 3526 raft_consensus.cc:3037] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.193773 3526 raft_consensus.cc:504] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.194306 3526 leader_election.cc:288] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.194712 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.194720 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.194926 3292 raft_consensus.cc:3037] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.194949 3441 raft_consensus.cc:3037] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.196894 3292 raft_consensus.cc:2454] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.196991 3441 raft_consensus.cc:2454] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.197261 3332 leader_election.cc:302] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.197511 3526 raft_consensus.cc:2781] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.197719 3526 raft_consensus.cc:686] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.197978 3526 consensus_queue.cc:237] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.200301 3190 catalog_manager.cc:5484] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 1 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.259665 3526 raft_consensus.cc:482] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.259881 3526 raft_consensus.cc:504] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.260545 3526 leader_election.cc:288] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.261035 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.261078 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.261230 3292 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.261268 3441 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.261623 3332 leader_election.cc:302] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.261875 3526 raft_consensus.cc:2781] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.261991 3526 raft_consensus.cc:482] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.262085 3526 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.263913 3526 raft_consensus.cc:504] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.264498 3526 leader_election.cc:288] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.264981 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.264950 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.265197 3441 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.265193 3292 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.268267 3292 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.268289 3441 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.268699 3331 leader_election.cc:302] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.268963 3526 raft_consensus.cc:2781] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.269109 3526 raft_consensus.cc:686] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.269403 3526 consensus_queue.cc:237] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.272029 3526 raft_consensus.cc:482] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.272121 3190 catalog_manager.cc:5484] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 1 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.272239 3526 raft_consensus.cc:504] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.272887 3526 leader_election.cc:288] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.273290 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.273519 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" is_pre_election: true
I20240503 02:13:04.273595 3292 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.273768 3441 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 0.
I20240503 02:13:04.274030 3332 leader_election.cc:302] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.274304 3526 raft_consensus.cc:2781] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.274435 3526 raft_consensus.cc:482] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.274565 3526 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.276582 3526 raft_consensus.cc:504] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.277204 3526 leader_election.cc:288] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.277432 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.277688 3292 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.277652 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.277871 3441 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.279750 3292 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.280031 3441 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 1.
I20240503 02:13:04.280113 3332 leader_election.cc:302] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.280361 3526 raft_consensus.cc:2781] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.280529 3526 raft_consensus.cc:686] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.280795 3526 consensus_queue.cc:237] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.282953 3190 catalog_manager.cc:5484] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 1 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.398495 3491 raft_consensus.cc:482] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:04.398746 3491 raft_consensus.cc:504] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.399471 3491 leader_election.cc:288] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:04.399986 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" is_pre_election: true
I20240503 02:13:04.400028 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451" is_pre_election: true
I20240503 02:13:04.400206 3368 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 0.
I20240503 02:13:04.400233 3292 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 0.
I20240503 02:13:04.400611 3405 leader_election.cc:302] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.400916 3491 raft_consensus.cc:2781] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:04.401018 3491 raft_consensus.cc:482] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:04.401132 3491 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.403095 3491 raft_consensus.cc:504] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.403669 3491 leader_election.cc:288] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:04.404031 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.404110 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.404192 3292 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.404314 3368 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:04.406272 3292 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 1.
I20240503 02:13:04.406338 3368 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 1.
I20240503 02:13:04.406687 3406 leader_election.cc:302] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.406944 3491 raft_consensus.cc:2781] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:04.407105 3491 raft_consensus.cc:686] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:04.407357 3491 consensus_queue.cc:237] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.410029 3190 catalog_manager.cc:5484] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 0 to 1, leader changed from <none> to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 1 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:04.454751 3491 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.459604 3491 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.485800 3515 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.490434 3523 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.501121 3368 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:59524
I20240503 02:13:04.501317 3368 raft_consensus.cc:595] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:04.507596 3441 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:56618
I20240503 02:13:04.507779 3441 raft_consensus.cc:595] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:04.509649 3441 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d451172292d243818f33c121270e4888"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:56618
I20240503 02:13:04.509811 3441 raft_consensus.cc:595] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:04.511655 3368 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:59524
I20240503 02:13:04.511809 3368 raft_consensus.cc:595] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:04.513697 3368 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:59524
I20240503 02:13:04.513854 3368 raft_consensus.cc:595] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:04.522361 3292 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:45044
I20240503 02:13:04.522559 3292 raft_consensus.cc:595] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Received request to transfer leadership to TS bcb3dc993fb340d19a30629ac22f6b0c
I20240503 02:13:04.524497 3292 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:45044
I20240503 02:13:04.524693 3292 raft_consensus.cc:595] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Received request to transfer leadership to TS bcb3dc993fb340d19a30629ac22f6b0c
I20240503 02:13:04.526600 3368 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:59524
I20240503 02:13:04.526759 3368 raft_consensus.cc:595] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Received request to transfer leadership to TS a57af4b5101e489bb3f4fecbd94bcd04
I20240503 02:13:04.570155 3523 consensus_queue.cc:1035] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.574139 3523 consensus_queue.cc:1035] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.576920 3488 raft_consensus.cc:979] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:04.577095 3523 raft_consensus.cc:1067] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Signalling peer bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:04.577684 3368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:59518
I20240503 02:13:04.577881 3368 raft_consensus.cc:482] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:04.578079 3368 raft_consensus.cc:3037] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.580116 3368 raft_consensus.cc:504] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.580682 3368 leader_election.cc:288] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 2 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.581012 3292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.581216 3292 raft_consensus.cc:3032] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:04.581301 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5abf63247bb47c294c1e7c45f6d26a0" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.581317 3292 raft_consensus.cc:729] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.581502 3441 raft_consensus.cc:3037] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.581717 3292 consensus_queue.cc:260] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.582135 3292 raft_consensus.cc:3037] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.583906 3441 raft_consensus.cc:2454] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 2.
I20240503 02:13:04.584153 3292 raft_consensus.cc:2454] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 2.
I20240503 02:13:04.584475 3331 leader_election.cc:302] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.585327 3526 raft_consensus.cc:2781] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:04.585449 3526 raft_consensus.cc:686] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.585740 3526 consensus_queue.cc:237] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.587707 3190 catalog_manager.cc:5484] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 1 to 2, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 2 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.603051 3526 consensus_queue.cc:1035] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.608021 3526 consensus_queue.cc:1035] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.633216 3526 consensus_queue.cc:1035] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.637287 3526 consensus_queue.cc:1035] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.640125 3489 raft_consensus.cc:979] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.640266 3489 raft_consensus.cc:1067] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.640836 3292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45016
I20240503 02:13:04.641016 3292 raft_consensus.cc:482] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:04.641129 3292 raft_consensus.cc:3037] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.643222 3292 raft_consensus.cc:504] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.643754 3292 leader_election.cc:288] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.644168 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.644227 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d488e03df854d38830d4f004e7d3a4d" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.644335 3368 raft_consensus.cc:3032] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:04.644419 3368 raft_consensus.cc:729] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.644474 3441 raft_consensus.cc:3037] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.644713 3368 consensus_queue.cc:260] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.644959 3368 raft_consensus.cc:3037] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.646699 3441 raft_consensus.cc:2454] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.646939 3368 raft_consensus.cc:2454] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.647137 3256 leader_election.cc:302] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.647403 3523 raft_consensus.cc:2781] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:04.647539 3523 raft_consensus.cc:686] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.647800 3523 consensus_queue.cc:237] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.649749 3189 catalog_manager.cc:5484] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.671098 3489 consensus_queue.cc:1035] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.675175 3526 consensus_queue.cc:1035] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.702121 3526 consensus_queue.cc:1035] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.704743 3489 raft_consensus.cc:979] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.704989 3489 raft_consensus.cc:1067] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.705968 3293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45016
I20240503 02:13:04.706614 3293 raft_consensus.cc:482] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:04.706780 3293 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.706777 3489 consensus_queue.cc:1035] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.709831 3293 raft_consensus.cc:504] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.710769 3293 leader_election.cc:288] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.710799 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.711061 3368 raft_consensus.cc:3032] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:04.711156 3368 raft_consensus.cc:729] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.711340 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.711524 3441 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.711447 3368 consensus_queue.cc:260] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.711808 3368 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.714762 3441 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.714740 3368 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.715145 3256 leader_election.cc:302] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.715399 3523 raft_consensus.cc:2781] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:04.715581 3523 raft_consensus.cc:686] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.715828 3523 consensus_queue.cc:237] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.717751 3190 catalog_manager.cc:5484] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.758347 3526 consensus_queue.cc:1035] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.759263 3526 consensus_queue.cc:1035] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.763034 3489 consensus_queue.cc:1035] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.764998 3526 consensus_queue.cc:1035] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.766836 3529 raft_consensus.cc:979] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.767091 3526 raft_consensus.cc:1067] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.769690 3293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45016
I20240503 02:13:04.769940 3293 raft_consensus.cc:482] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:04.770053 3293 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.772018 3293 raft_consensus.cc:504] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.772497 3293 leader_election.cc:288] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.772964 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.773190 3368 raft_consensus.cc:3032] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:04.773281 3368 raft_consensus.cc:729] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.773612 3368 consensus_queue.cc:260] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.773901 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.773938 3368 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.774091 3441 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.776085 3368 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.776489 3256 leader_election.cc:302] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.776692 3441 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.776895 3488 raft_consensus.cc:2781] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:04.777097 3488 raft_consensus.cc:686] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.777339 3488 consensus_queue.cc:237] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.776918 3526 raft_consensus.cc:979] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:04.778501 3526 raft_consensus.cc:1067] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Signalling peer a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:04.779263 3441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:56604
I20240503 02:13:04.779459 3441 raft_consensus.cc:482] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:04.779594 3441 raft_consensus.cc:3037] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.779861 3189 catalog_manager.cc:5484] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.781565 3441 raft_consensus.cc:504] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.782085 3441 leader_election.cc:288] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 2 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:04.782527 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:04.782655 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ac4be8db7d546d3a8a75f14203d95ab" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.782722 3293 raft_consensus.cc:3037] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.782850 3368 raft_consensus.cc:3032] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:04.782929 3368 raft_consensus.cc:729] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:04.783191 3368 consensus_queue.cc:260] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.783522 3368 raft_consensus.cc:3037] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.784720 3293 raft_consensus.cc:2454] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 2.
I20240503 02:13:04.785040 3406 leader_election.cc:302] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:04.785280 3491 raft_consensus.cc:2781] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:04.785432 3491 raft_consensus.cc:686] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:04.785437 3368 raft_consensus.cc:2454] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 2.
I20240503 02:13:04.785707 3491 consensus_queue.cc:237] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.787578 3189 catalog_manager.cc:5484] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 1 to 2, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 2 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:04.903301 3491 consensus_queue.cc:1035] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.905983 3509 raft_consensus.cc:979] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.906242 3491 raft_consensus.cc:1067] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:04.907099 3291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45030
I20240503 02:13:04.907289 3291 raft_consensus.cc:482] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:04.907405 3291 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.908303 3491 consensus_queue.cc:1035] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:04.910154 3291 raft_consensus.cc:504] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.912252 3291 leader_election.cc:288] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:04.912710 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:04.912910 3368 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.913105 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:04.913347 3441 raft_consensus.cc:3032] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:04.913434 3441 raft_consensus.cc:729] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:04.913725 3441 consensus_queue.cc:260] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.914073 3441 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:04.915411 3368 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.915732 3256 leader_election.cc:302] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:04.915967 3488 raft_consensus.cc:2781] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:04.916128 3488 raft_consensus.cc:686] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:04.916179 3441 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:04.916404 3488 consensus_queue.cc:237] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:04.918263 3190 catalog_manager.cc:5484] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:04.964015 3441 raft_consensus.cc:1261] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:04.964655 3529 consensus_queue.cc:1035] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:04.967723 3291 raft_consensus.cc:1261] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:04.968366 3489 consensus_queue.cc:1035] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.031421 3523 raft_consensus.cc:979] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:05.031620 3523 raft_consensus.cc:1067] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Signalling peer bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:05.032253 3368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:59518
I20240503 02:13:05.032447 3368 raft_consensus.cc:482] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:05.032584 3368 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:05.034664 3368 raft_consensus.cc:504] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:05.035457 3368 leader_election.cc:288] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 2 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:05.035841 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:05.036062 3441 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:05.036069 3491 raft_consensus.cc:979] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:05.036266 3491 raft_consensus.cc:1067] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:05.036340 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:05.036692 3293 raft_consensus.cc:3032] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:05.036863 3291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d451172292d243818f33c121270e4888"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45030
I20240503 02:13:05.036993 3291 raft_consensus.cc:482] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:05.037206 3291 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:05.039113 3291 raft_consensus.cc:504] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:05.039913 3291 leader_election.cc:288] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:05.040372 3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:05.040467 3293 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:05.040604 3440 raft_consensus.cc:3032] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:05.040724 3440 raft_consensus.cc:729] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:05.040848 3293 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:05.041023 3440 consensus_queue.cc:260] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:05.041285 3293 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:05.041371 3440 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:05.041999 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:05.042234 3368 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:05.044664 3293 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 2.
I20240503 02:13:05.044699 3440 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:05.045248 3332 leader_election.cc:302] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:05.045266 3256 leader_election.cc:302] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:05.045390 3441 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 2.
I20240503 02:13:05.045706 3489 raft_consensus.cc:2781] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:05.045886 3489 raft_consensus.cc:686] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:05.046258 3489 consensus_queue.cc:237] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:05.047159 3523 raft_consensus.cc:2781] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:05.047315 3523 raft_consensus.cc:686] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:05.047487 3368 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:05.047608 3523 consensus_queue.cc:237] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:05.049023 3190 catalog_manager.cc:5484] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 1 to 2, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 2 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:05.050177 3189 catalog_manager.cc:5484] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:05.060166 3441 raft_consensus.cc:1261] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.060621 3488 consensus_queue.cc:1035] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.063071 3368 raft_consensus.cc:1261] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.063586 3515 consensus_queue.cc:1035] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.114471 3441 raft_consensus.cc:1261] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.114971 3515 consensus_queue.cc:1035] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.117792 3368 raft_consensus.cc:1261] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.118537 3488 consensus_queue.cc:1035] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.287536 3441 raft_consensus.cc:1261] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.288182 3515 consensus_queue.cc:1035] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.291049 3368 raft_consensus.cc:1261] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.291774 3515 consensus_queue.cc:1035] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.301100 3368 raft_consensus.cc:1261] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.301599 3559 consensus_queue.cc:1035] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.304394 3293 raft_consensus.cc:1261] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.304929 3509 consensus_queue.cc:1035] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.404270 3368 raft_consensus.cc:1261] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.404893 3515 consensus_queue.cc:1035] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.407994 3441 raft_consensus.cc:1261] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.408816 3515 consensus_queue.cc:1035] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.448923 3368 raft_consensus.cc:1261] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.449477 3488 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.453043 3441 raft_consensus.cc:1261] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.453748 3515 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.479246 3293 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.479945 3526 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:05.483469 3441 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:05.483948 3489 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:07.537351 2677 auto_leader_rebalancer-test.cc:331] The leader distribution is
I20240503 02:13:07.537670 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:07.545166 3293 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d451172292d243818f33c121270e4888"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:45054
I20240503 02:13:07.545369 3293 raft_consensus.cc:595] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Received request to transfer leadership to TS a57af4b5101e489bb3f4fecbd94bcd04
I20240503 02:13:07.551432 3368 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:59540
I20240503 02:13:07.551621 3368 raft_consensus.cc:595] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Received request to transfer leadership to TS a57af4b5101e489bb3f4fecbd94bcd04
I20240503 02:13:07.552098 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 2
I20240503 02:13:07.552325 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:07.746374 3561 raft_consensus.cc:979] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:07.746572 3561 raft_consensus.cc:1067] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Signalling peer a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:07.747133 3441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:56604
I20240503 02:13:07.747335 3441 raft_consensus.cc:482] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:07.747443 3441 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:07.749361 3441 raft_consensus.cc:504] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:07.749881 3441 leader_election.cc:288] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 3 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:07.750321 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:07.750376 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:07.750515 3293 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:07.750594 3368 raft_consensus.cc:3032] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:13:07.750702 3368 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:07.750952 3368 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:07.751266 3368 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:07.752470 3293 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 3.
I20240503 02:13:07.752839 3406 leader_election.cc:302] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:07.753110 3563 raft_consensus.cc:2781] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:13:07.753225 3368 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 3.
I20240503 02:13:07.753645 3563 raft_consensus.cc:686] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:07.753947 3563 consensus_queue.cc:237] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:07.756183 3191 catalog_manager.cc:5484] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 2 to 3, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 3 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:07.991793 3574 raft_consensus.cc:979] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:07.991971 3574 raft_consensus.cc:1067] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Signalling peer a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:07.992558 3441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d451172292d243818f33c121270e4888"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:56608
I20240503 02:13:07.992712 3441 raft_consensus.cc:482] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:07.992800 3441 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:07.994652 3441 raft_consensus.cc:504] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:07.995139 3441 leader_election.cc:288] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 3 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:07.995463 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:07.995554 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:07.995644 3293 raft_consensus.cc:3032] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:13:07.995769 3293 raft_consensus.cc:729] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:07.995790 3368 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:07.996029 3293 consensus_queue.cc:260] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:07.996315 3293 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:07.998057 3368 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 3.
I20240503 02:13:07.998222 3293 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 3.
I20240503 02:13:07.998406 3405 leader_election.cc:302] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:07.998721 3563 raft_consensus.cc:2781] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:13:07.998898 3563 raft_consensus.cc:686] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:07.999126 3563 consensus_queue.cc:237] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:08.001731 3191 catalog_manager.cc:5484] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 2 to 3, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 3 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:08.147490 3368 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:08.148000 3563 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:08.151541 3293 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:08.152738 3568 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:08.493782 3293 raft_consensus.cc:1261] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:08.494290 3568 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:08.497629 3367 raft_consensus.cc:1261] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:08.498217 3582 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:08.552716 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:08.554354 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:08.554505 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:09.554913 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:09.556763 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:09.556978 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:10.557355 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:10.559468 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:10.559700 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:11.560030 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:11.561794 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:11.561970 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:12.562387 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:12.564112 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:12.564318 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:13.564684 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:13.566463 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:13.566624 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:14.567099 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:14.569407 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:14.569629 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:15.569996 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:15.571671 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:15.571822 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:16.572237 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:16.574023 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:16.574188 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:17.584546 2677 auto_leader_rebalancer-test.cc:350] The leader distribution is
I20240503 02:13:17.587925 3441 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d451172292d243818f33c121270e4888"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:56618
I20240503 02:13:17.588147 3441 raft_consensus.cc:595] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:17.592000 3367 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:59524
I20240503 02:13:17.592185 3367 raft_consensus.cc:595] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:17.593870 3367 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "837ab97c37244c3384d348e4b391b132"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:59524
I20240503 02:13:17.594055 3367 raft_consensus.cc:595] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:17.595727 3441 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
mode: GRACEFUL
new_leader_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:56618
I20240503 02:13:17.595889 3441 raft_consensus.cc:595] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Received request to transfer leadership to TS 2403b5a86c334221bd4caec2a6ad6451
I20240503 02:13:17.663125 3586 raft_consensus.cc:979] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:17.663278 3586 raft_consensus.cc:1067] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:17.663887 3293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45016
I20240503 02:13:17.664100 3293 raft_consensus.cc:482] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:17.664219 3293 raft_consensus.cc:3037] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:17.666152 3293 raft_consensus.cc:504] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:17.666680 3293 leader_election.cc:288] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:17.667112 3367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:17.667152 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9040d9c44fa4791a3f5e17dcb896f32" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:17.667327 3367 raft_consensus.cc:3032] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:17.667407 3441 raft_consensus.cc:3037] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:17.667423 3367 raft_consensus.cc:729] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:17.667714 3367 consensus_queue.cc:260] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:17.668021 3367 raft_consensus.cc:3037] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:17.669517 3441 raft_consensus.cc:2454] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:17.669962 3256 leader_election.cc:302] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:17.670035 3367 raft_consensus.cc:2454] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:17.670274 3574 raft_consensus.cc:2781] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:17.670632 3574 raft_consensus.cc:686] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:17.670939 3574 consensus_queue.cc:237] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:17.673319 3190 catalog_manager.cc:5484] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:17.897357 3561 raft_consensus.cc:979] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:17.897559 3561 raft_consensus.cc:1067] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:17.898156 3293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "837ab97c37244c3384d348e4b391b132"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45016
I20240503 02:13:17.898355 3293 raft_consensus.cc:482] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:17.898459 3293 raft_consensus.cc:3037] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:17.900450 3293 raft_consensus.cc:504] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:17.901021 3293 leader_election.cc:288] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:17.901492 3367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837ab97c37244c3384d348e4b391b132" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:17.901679 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837ab97c37244c3384d348e4b391b132" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:17.901866 3441 raft_consensus.cc:3037] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:17.901844 3367 raft_consensus.cc:3032] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:17.902017 3367 raft_consensus.cc:729] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:17.902325 3367 consensus_queue.cc:260] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:17.902768 3367 raft_consensus.cc:3037] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:17.905072 3441 raft_consensus.cc:2454] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:17.905520 3256 leader_election.cc:302] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, a57af4b5101e489bb3f4fecbd94bcd04; no voters:
I20240503 02:13:17.905740 3367 raft_consensus.cc:2454] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 2.
I20240503 02:13:17.905864 3574 raft_consensus.cc:2781] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:17.906033 3574 raft_consensus.cc:686] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:17.906296 3574 consensus_queue.cc:237] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:17.908665 3190 catalog_manager.cc:5484] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 1 to 2, leader changed from bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 2 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:18.061038 3585 raft_consensus.cc:979] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:18.061098 3595 raft_consensus.cc:979] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: : Instructing follower 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:18.061267 3585 raft_consensus.cc:1067] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:18.061329 3594 raft_consensus.cc:1067] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Signalling peer 2403b5a86c334221bd4caec2a6ad6451 to start an election
I20240503 02:13:18.061858 3291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d451172292d243818f33c121270e4888"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45030
I20240503 02:13:18.062011 3293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
from {username='slave'} at 127.0.0.1:45030
I20240503 02:13:18.062083 3291 raft_consensus.cc:482] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:18.062186 3293 raft_consensus.cc:482] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:18.062203 3291 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:13:18.062322 3293 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:13:18.064219 3293 raft_consensus.cc:504] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:18.064342 3291 raft_consensus.cc:504] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:18.064858 3293 leader_election.cc:288] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 4 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:18.064920 3291 leader_election.cc:288] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 4 election: Requested vote from peers bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:18.065425 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:18.065490 3367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:18.065670 3368 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:13:18.065445 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:18.065771 3367 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:13:18.065946 3441 raft_consensus.cc:3032] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Stepping down as leader of term 3
I20240503 02:13:18.066078 3441 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:18.066022 3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d451172292d243818f33c121270e4888" candidate_uuid: "2403b5a86c334221bd4caec2a6ad6451" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:18.066270 3440 raft_consensus.cc:3032] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Stepping down as leader of term 3
I20240503 02:13:18.066392 3440 raft_consensus.cc:729] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:18.066454 3441 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:18.066722 3440 consensus_queue.cc:260] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:18.066905 3441 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:13:18.067025 3440 raft_consensus.cc:3037] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:13:18.068996 3367 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 4.
I20240503 02:13:18.069078 3368 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 4.
I20240503 02:13:18.069406 3256 leader_election.cc:302] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:18.069674 3574 raft_consensus.cc:2781] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 02:13:18.069720 3256 leader_election.cc:302] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2403b5a86c334221bd4caec2a6ad6451, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:18.069937 3440 raft_consensus.cc:2454] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 4.
I20240503 02:13:18.069998 3587 raft_consensus.cc:2781] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 02:13:18.070019 3441 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2403b5a86c334221bd4caec2a6ad6451 in term 4.
I20240503 02:13:18.070231 3587 raft_consensus.cc:686] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:18.070609 3587 consensus_queue.cc:237] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:18.072459 3574 raft_consensus.cc:686] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Becoming Leader. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:18.072842 3574 consensus_queue.cc:237] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:18.078409 3190 catalog_manager.cc:5484] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 3 to 4, leader changed from a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 4 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:18.082756 3190 catalog_manager.cc:5484] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 reported cstate change: term changed from 3 to 4, leader changed from a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67) to 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65). New cstate: current_term: 4 leader_uuid: "2403b5a86c334221bd4caec2a6ad6451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:18.100387 3368 raft_consensus.cc:1261] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:18.100852 3574 consensus_queue.cc:1035] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:18.104614 3441 raft_consensus.cc:1261] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:18.105144 3597 consensus_queue.cc:1035] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:18.452915 3441 raft_consensus.cc:1261] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:18.453498 3597 consensus_queue.cc:1035] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:18.456909 3368 raft_consensus.cc:1261] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240503 02:13:18.457845 3441 raft_consensus.cc:1261] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 02:13:18.458000 3597 consensus_queue.cc:1035] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:18.458825 3597 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:18.463678 3368 raft_consensus.cc:1261] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 02:13:18.464367 3574 consensus_queue.cc:1035] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:18.654903 3441 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 02:13:18.655630 3574 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:18.660465 3368 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Refusing update from remote peer 2403b5a86c334221bd4caec2a6ad6451: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20240503 02:13:18.661067 3587 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:20.615927 2677 auto_leader_rebalancer-test.cc:365] The leader distribution is
I20240503 02:13:20.616197 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:20.618484 3291 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:45054
I20240503 02:13:20.618681 3291 raft_consensus.cc:595] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Received request to transfer leadership to TS bcb3dc993fb340d19a30629ac22f6b0c
I20240503 02:13:20.619699 3291 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:45054
I20240503 02:13:20.619869 3291 raft_consensus.cc:595] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Received request to transfer leadership to TS a57af4b5101e489bb3f4fecbd94bcd04
I20240503 02:13:20.620813 3291 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:45054
I20240503 02:13:20.620975 3291 raft_consensus.cc:595] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Received request to transfer leadership to TS bcb3dc993fb340d19a30629ac22f6b0c
I20240503 02:13:20.621846 3291 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
mode: GRACEFUL
new_leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:45054
I20240503 02:13:20.622007 3291 raft_consensus.cc:595] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Received request to transfer leadership to TS a57af4b5101e489bb3f4fecbd94bcd04
I20240503 02:13:20.622377 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 4
I20240503 02:13:20.622584 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:20.780273 3587 raft_consensus.cc:979] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:20.780419 3587 raft_consensus.cc:1067] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Signalling peer a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:20.781021 3441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:56608
I20240503 02:13:20.781190 3441 raft_consensus.cc:482] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:20.781281 3441 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Advancing to term 5
I20240503 02:13:20.783169 3441 raft_consensus.cc:504] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:20.783782 3441 leader_election.cc:288] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 5 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:20.784186 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:20.784163 3291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e356847e2c24487aaa5a99f3071efecd" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:20.784449 3291 raft_consensus.cc:3032] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Stepping down as leader of term 4
I20240503 02:13:20.784457 3368 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Advancing to term 5
I20240503 02:13:20.784569 3291 raft_consensus.cc:729] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:20.784844 3291 consensus_queue.cc:260] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:20.785162 3291 raft_consensus.cc:3037] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 4 FOLLOWER]: Advancing to term 5
I20240503 02:13:20.786545 3368 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 5.
I20240503 02:13:20.786942 3405 leader_election.cc:302] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:20.787185 3291 raft_consensus.cc:2454] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 5.
I20240503 02:13:20.787240 3613 raft_consensus.cc:2781] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 5 FOLLOWER]: Leader election won for term 5
I20240503 02:13:20.787779 3613 raft_consensus.cc:686] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 5 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:20.788064 3613 consensus_queue.cc:237] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:20.790671 3191 catalog_manager.cc:5484] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 4 to 5, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 5 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:20.866618 3614 raft_consensus.cc:979] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:20.866791 3614 raft_consensus.cc:1067] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Signalling peer bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:20.867475 3368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:59518
I20240503 02:13:20.867652 3368 raft_consensus.cc:482] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:20.867751 3368 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:20.869655 3368 raft_consensus.cc:504] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:20.870354 3368 leader_election.cc:288] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 3 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:20.870769 3291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:20.870790 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc754ea99c59493abae00c50d74c3e02" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:20.870930 3291 raft_consensus.cc:3032] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:13:20.870981 3441 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:20.871043 3291 raft_consensus.cc:729] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:20.871299 3291 consensus_queue.cc:260] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:20.871608 3291 raft_consensus.cc:3037] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:20.872954 3441 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 3.
I20240503 02:13:20.873311 3331 leader_election.cc:302] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:20.873569 3612 raft_consensus.cc:2781] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:13:20.873626 3291 raft_consensus.cc:2454] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 3.
I20240503 02:13:20.874056 3612 raft_consensus.cc:686] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:20.874356 3612 consensus_queue.cc:237] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:20.876794 3191 catalog_manager.cc:5484] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 2 to 3, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 3 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:21.061295 3614 raft_consensus.cc:979] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:21.061487 3614 raft_consensus.cc:1067] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Signalling peer bcb3dc993fb340d19a30629ac22f6b0c to start an election
I20240503 02:13:21.062148 3368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a"
dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
from {username='slave'} at 127.0.0.1:59518
I20240503 02:13:21.062361 3368 raft_consensus.cc:482] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:21.062460 3368 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:21.064350 3368 raft_consensus.cc:504] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:21.064882 3368 leader_election.cc:288] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 3 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67:33707)
I20240503 02:13:21.065299 3441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
I20240503 02:13:21.065325 3291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39eeea4f75de433eb3e1f744b3f4e31a" candidate_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:21.065558 3291 raft_consensus.cc:3032] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:13:21.065559 3441 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:21.065721 3291 raft_consensus.cc:729] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:21.065999 3291 consensus_queue.cc:260] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:21.066409 3291 raft_consensus.cc:3037] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:21.068524 3441 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 3.
I20240503 02:13:21.068889 3331 leader_election.cc:302] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:21.068929 3291 raft_consensus.cc:2454] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb3dc993fb340d19a30629ac22f6b0c in term 3.
I20240503 02:13:21.069176 3612 raft_consensus.cc:2781] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:13:21.069351 3612 raft_consensus.cc:686] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 3 LEADER]: Becoming Leader. State: Replica: bcb3dc993fb340d19a30629ac22f6b0c, State: Running, Role: LEADER
I20240503 02:13:21.069645 3612 consensus_queue.cc:237] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:21.072082 3191 catalog_manager.cc:5484] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c reported cstate change: term changed from 2 to 3, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66). New cstate: current_term: 3 leader_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:21.160176 3587 raft_consensus.cc:979] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: : Instructing follower a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:21.160395 3587 raft_consensus.cc:1067] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Signalling peer a57af4b5101e489bb3f4fecbd94bcd04 to start an election
I20240503 02:13:21.161005 3441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76"
dest_uuid: "a57af4b5101e489bb3f4fecbd94bcd04"
from {username='slave'} at 127.0.0.1:56608
I20240503 02:13:21.161173 3441 raft_consensus.cc:482] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240503 02:13:21.161299 3441 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:21.163200 3441 raft_consensus.cc:504] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:21.163781 3441 leader_election.cc:288] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 3 election: Requested vote from peers 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163), bcb3dc993fb340d19a30629ac22f6b0c (127.2.157.66:42373)
I20240503 02:13:21.164177 3368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb3dc993fb340d19a30629ac22f6b0c"
I20240503 02:13:21.164202 3291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b5f97b3837f454d824325797e30ae76" candidate_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2403b5a86c334221bd4caec2a6ad6451"
I20240503 02:13:21.164456 3368 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:21.164462 3291 raft_consensus.cc:3032] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Stepping down as leader of term 2
I20240503 02:13:21.164613 3291 raft_consensus.cc:729] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2403b5a86c334221bd4caec2a6ad6451, State: Running, Role: LEADER
I20240503 02:13:21.164886 3291 consensus_queue.cc:260] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:21.165273 3291 raft_consensus.cc:3037] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:21.167484 3368 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 3.
I20240503 02:13:21.167903 3405 leader_election.cc:302] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57af4b5101e489bb3f4fecbd94bcd04, bcb3dc993fb340d19a30629ac22f6b0c; no voters:
I20240503 02:13:21.167997 3291 raft_consensus.cc:2454] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57af4b5101e489bb3f4fecbd94bcd04 in term 3.
I20240503 02:13:21.168176 3613 raft_consensus.cc:2781] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:13:21.168359 3613 raft_consensus.cc:686] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Becoming Leader. State: Replica: a57af4b5101e489bb3f4fecbd94bcd04, State: Running, Role: LEADER
I20240503 02:13:21.168638 3613 consensus_queue.cc:237] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } }
I20240503 02:13:21.170914 3191 catalog_manager.cc:5484] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 reported cstate change: term changed from 2 to 3, leader changed from 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65) to a57af4b5101e489bb3f4fecbd94bcd04 (127.2.157.67). New cstate: current_term: 3 leader_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:21.179172 3368 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 5 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240503 02:13:21.179661 3613 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240503 02:13:21.183991 3291 raft_consensus.cc:1261] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 5 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20240503 02:13:21.184504 3613 consensus_queue.cc:1035] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240503 02:13:21.281265 3291 raft_consensus.cc:1261] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:21.281841 3612 consensus_queue.cc:1035] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:21.285387 3441 raft_consensus.cc:1261] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:21.285940 3612 consensus_queue.cc:1035] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:21.544353 3441 raft_consensus.cc:1261] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:21.544962 3612 consensus_queue.cc:1035] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57af4b5101e489bb3f4fecbd94bcd04" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:21.549057 3291 raft_consensus.cc:1261] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Refusing update from remote peer bcb3dc993fb340d19a30629ac22f6b0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:21.549602 3612 consensus_queue.cc:1035] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:21.623019 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:21.624774 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:21.624948 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:21.772074 3368 raft_consensus.cc:1261] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:21.772589 3613 consensus_queue.cc:1035] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb3dc993fb340d19a30629ac22f6b0c" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:21.776041 3291 raft_consensus.cc:1261] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Refusing update from remote peer a57af4b5101e489bb3f4fecbd94bcd04: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:21.776660 3613 consensus_queue.cc:1035] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2403b5a86c334221bd4caec2a6ad6451" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:22.625401 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:22.627646 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:22.627817 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:23.628254 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:23.630272 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:23.630486 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:24.630932 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:24.632673 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:24.632876 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:25.633371 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:25.635136 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:25.635344 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:26.635782 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:26.637594 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:26.637784 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:27.638218 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:27.639979 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:27.640213 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:28.640643 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:28.642395 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:28.642585 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:29.642951 2677 auto_leader_rebalancer.cc:131] leader rebalance for table test-workload
I20240503 02:13:29.645287 2677 auto_leader_rebalancer.cc:379] table: test-workload, leader rebalance finish, leader transfer count: 0
I20240503 02:13:29.645490 2677 auto_leader_rebalancer.cc:437] All tables' leader rebalancing finished this round
I20240503 02:13:30.655730 2677 auto_leader_rebalancer-test.cc:376] The leader distribution is
I20240503 02:13:30.655947 2677 tablet_server.cc:178] TabletServer@127.2.157.65:0 shutting down...
W20240503 02:13:30.667358 3406 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20240503 02:13:30.668081 2677 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 02:13:30.668546 2677 tablet_replica.cc:330] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.668743 2677 raft_consensus.cc:2229] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus shutting down.
W20240503 02:13:30.668859 3406 consensus_peers.cc:485] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 -> Peer 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163): Couldn't send request to peer 2403b5a86c334221bd4caec2a6ad6451. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:30.668954 2677 raft_consensus.cc:2258] T 2ac4be8db7d546d3a8a75f14203d95ab P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.669492 2677 tablet_replica.cc:330] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.669716 2677 raft_consensus.cc:2229] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.669874 2677 raft_consensus.cc:2258] T b5abf63247bb47c294c1e7c45f6d26a0 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.670323 2677 tablet_replica.cc:330] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.670477 2677 raft_consensus.cc:2229] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.670637 2677 raft_consensus.cc:2258] T 39eeea4f75de433eb3e1f744b3f4e31a P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.671068 2677 tablet_replica.cc:330] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.671224 2677 raft_consensus.cc:2229] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.671358 2677 raft_consensus.cc:2258] T bc754ea99c59493abae00c50d74c3e02 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.671766 2677 tablet_replica.cc:330] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.671907 2677 raft_consensus.cc:2229] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.672058 2677 raft_consensus.cc:2258] T 8b5f97b3837f454d824325797e30ae76 P 2403b5a86c334221bd4caec2a6ad6451 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.672451 2677 tablet_replica.cc:330] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.672608 2677 raft_consensus.cc:2229] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 4 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.672852 2677 raft_consensus.cc:2258] T d451172292d243818f33c121270e4888 P 2403b5a86c334221bd4caec2a6ad6451 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.673269 2677 tablet_replica.cc:330] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.673399 2677 raft_consensus.cc:2229] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 5 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.673557 2677 raft_consensus.cc:2258] T e356847e2c24487aaa5a99f3071efecd P 2403b5a86c334221bd4caec2a6ad6451 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.673946 2677 tablet_replica.cc:330] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.674093 2677 raft_consensus.cc:2229] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.674309 2677 raft_consensus.cc:2258] T 837ab97c37244c3384d348e4b391b132 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.674729 2677 tablet_replica.cc:330] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.674860 2677 raft_consensus.cc:2229] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.675133 2677 raft_consensus.cc:2258] T 8d488e03df854d38830d4f004e7d3a4d P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.675561 2677 tablet_replica.cc:330] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451: stopping tablet replica
I20240503 02:13:30.675726 2677 raft_consensus.cc:2229] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.675957 2677 raft_consensus.cc:2258] T b9040d9c44fa4791a3f5e17dcb896f32 P 2403b5a86c334221bd4caec2a6ad6451 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.691746 2677 tablet_server.cc:194] TabletServer@127.2.157.65:0 shutdown complete.
I20240503 02:13:30.704828 2677 tablet_server.cc:178] TabletServer@127.2.157.66:0 shutting down...
I20240503 02:13:30.714179 2677 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 02:13:30.714422 2677 tablet_replica.cc:330] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.714646 2677 raft_consensus.cc:2229] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 5 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.714813 2677 raft_consensus.cc:2258] T e356847e2c24487aaa5a99f3071efecd P bcb3dc993fb340d19a30629ac22f6b0c [term 5 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.715358 2677 tablet_replica.cc:330] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.715471 2677 raft_consensus.cc:2229] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.715581 2677 raft_consensus.cc:2258] T b9040d9c44fa4791a3f5e17dcb896f32 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.715938 2677 tablet_replica.cc:330] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.716029 2677 raft_consensus.cc:2229] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.716131 2677 raft_consensus.cc:2258] T 8d488e03df854d38830d4f004e7d3a4d P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.716454 2677 tablet_replica.cc:330] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.716547 2677 raft_consensus.cc:2229] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.716642 2677 raft_consensus.cc:2258] T d451172292d243818f33c121270e4888 P bcb3dc993fb340d19a30629ac22f6b0c [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.716956 2677 tablet_replica.cc:330] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.717047 2677 raft_consensus.cc:2229] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.717259 2677 raft_consensus.cc:2258] T 39eeea4f75de433eb3e1f744b3f4e31a P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.717648 2677 tablet_replica.cc:330] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.717741 2677 raft_consensus.cc:2229] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.717921 2677 raft_consensus.cc:2258] T b5abf63247bb47c294c1e7c45f6d26a0 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.718266 2677 tablet_replica.cc:330] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.718374 2677 raft_consensus.cc:2229] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.718498 2677 raft_consensus.cc:2258] T 2ac4be8db7d546d3a8a75f14203d95ab P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.718873 2677 tablet_replica.cc:330] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.718991 2677 raft_consensus.cc:2229] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.719110 2677 raft_consensus.cc:2258] T 837ab97c37244c3384d348e4b391b132 P bcb3dc993fb340d19a30629ac22f6b0c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.719468 2677 tablet_replica.cc:330] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.719617 2677 raft_consensus.cc:2229] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.719830 2677 raft_consensus.cc:2258] T bc754ea99c59493abae00c50d74c3e02 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.720218 2677 tablet_replica.cc:330] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c: stopping tablet replica
I20240503 02:13:30.720335 2677 raft_consensus.cc:2229] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.720458 2677 raft_consensus.cc:2258] T 8b5f97b3837f454d824325797e30ae76 P bcb3dc993fb340d19a30629ac22f6b0c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.735458 2677 tablet_server.cc:194] TabletServer@127.2.157.66:0 shutdown complete.
I20240503 02:13:30.739558 2677 tablet_server.cc:178] TabletServer@127.2.157.67:0 shutting down...
W20240503 02:13:30.746750 3406 consensus_peers.cc:485] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 -> Peer 2403b5a86c334221bd4caec2a6ad6451 (127.2.157.65:43163): Couldn't send request to peer 2403b5a86c334221bd4caec2a6ad6451. Status: Network error: Client connection negotiation failed: client connection to 127.2.157.65:43163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:30.749763 2677 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240503 02:13:30.750007 2677 tablet_replica.cc:330] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.750182 2677 raft_consensus.cc:2229] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.750344 2677 raft_consensus.cc:2258] T b5abf63247bb47c294c1e7c45f6d26a0 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.750859 2677 tablet_replica.cc:330] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.750998 2677 raft_consensus.cc:2229] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.751241 2677 raft_consensus.cc:2258] T 2ac4be8db7d546d3a8a75f14203d95ab P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.751628 2677 tablet_replica.cc:330] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.751753 2677 raft_consensus.cc:2229] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.751874 2677 raft_consensus.cc:2258] T 39eeea4f75de433eb3e1f744b3f4e31a P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.752242 2677 tablet_replica.cc:330] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.752367 2677 raft_consensus.cc:2229] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.752478 2677 raft_consensus.cc:2258] T d451172292d243818f33c121270e4888 P a57af4b5101e489bb3f4fecbd94bcd04 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.752868 2677 tablet_replica.cc:330] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.752990 2677 raft_consensus.cc:2229] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.753119 2677 raft_consensus.cc:2258] T bc754ea99c59493abae00c50d74c3e02 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.753507 2677 tablet_replica.cc:330] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.753672 2677 raft_consensus.cc:2229] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.753873 2677 raft_consensus.cc:2258] T 8b5f97b3837f454d824325797e30ae76 P a57af4b5101e489bb3f4fecbd94bcd04 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.754261 2677 tablet_replica.cc:330] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.754377 2677 raft_consensus.cc:2229] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 5 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.754588 2677 raft_consensus.cc:2258] T e356847e2c24487aaa5a99f3071efecd P a57af4b5101e489bb3f4fecbd94bcd04 [term 5 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.754949 2677 tablet_replica.cc:330] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.755072 2677 raft_consensus.cc:2229] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.755187 2677 raft_consensus.cc:2258] T 837ab97c37244c3384d348e4b391b132 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.755543 2677 tablet_replica.cc:330] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.755659 2677 raft_consensus.cc:2229] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.755774 2677 raft_consensus.cc:2258] T 8d488e03df854d38830d4f004e7d3a4d P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.756156 2677 tablet_replica.cc:330] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04: stopping tablet replica
I20240503 02:13:30.756268 2677 raft_consensus.cc:2229] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:30.756392 2677 raft_consensus.cc:2258] T b9040d9c44fa4791a3f5e17dcb896f32 P a57af4b5101e489bb3f4fecbd94bcd04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.771555 2677 tablet_server.cc:194] TabletServer@127.2.157.67:0 shutdown complete.
I20240503 02:13:30.780365 2677 master.cc:537] Master@127.2.157.126:33835 shutting down...
I20240503 02:13:30.787943 2677 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:13:30.788128 2677 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:30.788228 2677 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5a25ed62a3034c379554a1dfefa73cd2: stopping tablet replica
I20240503 02:13:30.801759 2677 master.cc:552] Master@127.2.157.126:33835 shutdown complete.
2403b5a86c334221bd4caec2a6ad6451 5
a57af4b5101e489bb3f4fecbd94bcd04 1
bcb3dc993fb340d19a30629ac22f6b0c 4
2403b5a86c334221bd4caec2a6ad6451 4
a57af4b5101e489bb3f4fecbd94bcd04 3
bcb3dc993fb340d19a30629ac22f6b0c 3
2403b5a86c334221bd4caec2a6ad6451 8
a57af4b5101e489bb3f4fecbd94bcd04 1
bcb3dc993fb340d19a30629ac22f6b0c 1
2403b5a86c334221bd4caec2a6ad6451 4
a57af4b5101e489bb3f4fecbd94bcd04 3
bcb3dc993fb340d19a30629ac22f6b0c 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (27378 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20240503 02:13:30.817099 2677 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.157.126:40759
I20240503 02:13:30.817597 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:30.820753 3646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:30.821492 3647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:30.822154 3649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:30.822786 2677 server_base.cc:1034] running on GCE node
I20240503 02:13:30.823014 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:30.823065 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:30.823099 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702410823094 us; error 0 us; skew 500 ppm
I20240503 02:13:30.823239 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:30.824488 2677 webserver.cc:460] Webserver started at http://127.2.157.126:38105/ using document root <none> and password file <none>
I20240503 02:13:30.824702 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:30.824774 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:30.824901 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:30.825485 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/master-0-root/instance:
uuid: "de3a7815a7214598a39277162a0cd94c"
format_stamp: "Formatted at 2024-05-03 02:13:30 on dist-test-slave-tz5s"
I20240503 02:13:30.827854 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:30.829282 3654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:30.829852 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20240503 02:13:30.829952 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/master-0-root
uuid: "de3a7815a7214598a39277162a0cd94c"
format_stamp: "Formatted at 2024-05-03 02:13:30 on dist-test-slave-tz5s"
I20240503 02:13:30.830060 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:30.844199 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:30.844669 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:30.856197 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.126:40759
I20240503 02:13:30.856215 3705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.126:40759 every 8 connection(s)
I20240503 02:13:30.857384 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:30.861403 3706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c: Bootstrap starting.
I20240503 02:13:30.862602 3706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:30.864027 3706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c: No bootstrap required, opened a new log
I20240503 02:13:30.864568 3706 raft_consensus.cc:348] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a7815a7214598a39277162a0cd94c" member_type: VOTER }
I20240503 02:13:30.864686 3706 raft_consensus.cc:374] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:30.864734 3706 raft_consensus.cc:729] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3a7815a7214598a39277162a0cd94c, State: Initialized, Role: FOLLOWER
I20240503 02:13:30.864910 3706 consensus_queue.cc:260] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a7815a7214598a39277162a0cd94c" member_type: VOTER }
I20240503 02:13:30.865021 3706 raft_consensus.cc:388] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:13:30.865073 3706 raft_consensus.cc:482] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:13:30.865150 3706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:30.866914 3706 raft_consensus.cc:504] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a7815a7214598a39277162a0cd94c" member_type: VOTER }
I20240503 02:13:30.867082 3706 leader_election.cc:302] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de3a7815a7214598a39277162a0cd94c; no voters:
I20240503 02:13:30.867424 3706 leader_election.cc:288] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:13:30.867529 3709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:30.867947 3709 raft_consensus.cc:686] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [term 1 LEADER]: Becoming Leader. State: Replica: de3a7815a7214598a39277162a0cd94c, State: Running, Role: LEADER
I20240503 02:13:30.868146 3709 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a7815a7214598a39277162a0cd94c" member_type: VOTER }
I20240503 02:13:30.868253 3706 sys_catalog.cc:564] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:13:30.868932 3711 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3a7815a7214598a39277162a0cd94c. Latest consensus state: current_term: 1 leader_uuid: "de3a7815a7214598a39277162a0cd94c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a7815a7214598a39277162a0cd94c" member_type: VOTER } }
I20240503 02:13:30.869071 3711 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [sys.catalog]: This master's current role is: LEADER
I20240503 02:13:30.869098 3710 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de3a7815a7214598a39277162a0cd94c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a7815a7214598a39277162a0cd94c" member_type: VOTER } }
I20240503 02:13:30.869275 3710 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c [sys.catalog]: This master's current role is: LEADER
I20240503 02:13:30.870065 3714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:13:30.871294 3714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:13:30.873297 2677 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240503 02:13:30.874110 3714 catalog_manager.cc:1347] Generated new cluster ID: aeb215c4678b4289912d35f4203806b3
I20240503 02:13:30.874198 3714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:13:30.888795 3714 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:13:30.889510 3714 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:13:30.895411 3714 catalog_manager.cc:5857] T 00000000000000000000000000000000 P de3a7815a7214598a39277162a0cd94c: Generated new TSK 0
I20240503 02:13:30.895613 3714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:13:30.905959 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:30.908980 3727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:30.909708 3728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:30.910320 3730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:30.910995 2677 server_base.cc:1034] running on GCE node
I20240503 02:13:30.911224 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:30.911288 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:30.911322 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702410911318 us; error 0 us; skew 500 ppm
I20240503 02:13:30.911453 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:30.912600 2677 webserver.cc:460] Webserver started at http://127.2.157.65:34179/ using document root <none> and password file <none>
I20240503 02:13:30.912812 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:30.912880 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:30.913003 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:30.913643 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root/instance:
uuid: "6e0d1128f0404d609df72fd607c20650"
format_stamp: "Formatted at 2024-05-03 02:13:30 on dist-test-slave-tz5s"
I20240503 02:13:30.915933 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:30.917239 3735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:30.917594 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20240503 02:13:30.917692 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root
uuid: "6e0d1128f0404d609df72fd607c20650"
format_stamp: "Formatted at 2024-05-03 02:13:30 on dist-test-slave-tz5s"
I20240503 02:13:30.917791 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:30.934696 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:30.935199 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:30.935736 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:30.936472 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:30.936520 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:30.936566 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:30.936594 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:30.949162 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.65:33841
I20240503 02:13:30.949193 3797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.65:33841 every 8 connection(s)
I20240503 02:13:30.950716 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:30.953869 3802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:30.954663 3803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:30.955250 3805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:30.955940 2677 server_base.cc:1034] running on GCE node
I20240503 02:13:30.956171 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:30.956214 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:30.956234 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702410956231 us; error 0 us; skew 500 ppm
I20240503 02:13:30.956310 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:30.957443 3798 heartbeater.cc:347] Connected to a master server at 127.2.157.126:40759
I20240503 02:13:30.957580 3798 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:30.957710 2677 webserver.cc:460] Webserver started at http://127.2.157.66:40225/ using document root <none> and password file <none>
I20240503 02:13:30.957885 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:30.957926 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:30.957911 3798 heartbeater.cc:511] Master 127.2.157.126:40759 requested a full tablet report, sending...
I20240503 02:13:30.958019 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:30.958664 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root/instance:
uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
format_stamp: "Formatted at 2024-05-03 02:13:30 on dist-test-slave-tz5s"
I20240503 02:13:30.958725 3671 ts_manager.cc:194] Registered new tserver with Master: 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841)
I20240503 02:13:30.960227 3671 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54330
I20240503 02:13:30.961517 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20240503 02:13:30.962833 3810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:30.963107 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20240503 02:13:30.963183 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root
uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
format_stamp: "Formatted at 2024-05-03 02:13:30 on dist-test-slave-tz5s"
I20240503 02:13:30.963276 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:30.985879 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:30.986362 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:30.986863 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:30.987577 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:30.987635 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:30.987711 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:30.987749 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:31.000587 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.66:43979
I20240503 02:13:31.000612 3872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.66:43979 every 8 connection(s)
I20240503 02:13:31.002159 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:31.005684 3876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:31.006333 3877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:31.006980 2677 server_base.cc:1034] running on GCE node
W20240503 02:13:31.007685 3879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:31.008025 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20240503 02:13:31.008067 3873 heartbeater.cc:347] Connected to a master server at 127.2.157.126:40759
W20240503 02:13:31.008109 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:31.008138 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702411008135 us; error 0 us; skew 500 ppm
I20240503 02:13:31.008203 3873 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:31.008306 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:31.008574 3873 heartbeater.cc:511] Master 127.2.157.126:40759 requested a full tablet report, sending...
I20240503 02:13:31.009392 3671 ts_manager.cc:194] Registered new tserver with Master: 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:31.009645 2677 webserver.cc:460] Webserver started at http://127.2.157.67:33629/ using document root <none> and password file <none>
I20240503 02:13:31.009804 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:31.009847 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:31.009934 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:31.010118 3671 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54332
I20240503 02:13:31.010486 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root/instance:
uuid: "4001ccb96b064ad2a2ca9c61096516fe"
format_stamp: "Formatted at 2024-05-03 02:13:31 on dist-test-slave-tz5s"
I20240503 02:13:31.012804 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:31.014147 3884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:31.014401 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20240503 02:13:31.014504 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root
uuid: "4001ccb96b064ad2a2ca9c61096516fe"
format_stamp: "Formatted at 2024-05-03 02:13:31 on dist-test-slave-tz5s"
I20240503 02:13:31.014585 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:31.028616 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:31.029021 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:31.029466 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:31.030295 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:31.030349 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:31.030405 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:31.030452 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:31.044787 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.67:34983
I20240503 02:13:31.044814 3946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.67:34983 every 8 connection(s)
I20240503 02:13:31.051146 3947 heartbeater.cc:347] Connected to a master server at 127.2.157.126:40759
I20240503 02:13:31.051251 3947 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:31.051535 3947 heartbeater.cc:511] Master 127.2.157.126:40759 requested a full tablet report, sending...
I20240503 02:13:31.052245 3671 ts_manager.cc:194] Registered new tserver with Master: 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:31.052788 2677 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006787488s
I20240503 02:13:31.052881 3671 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54340
I20240503 02:13:31.053722 2677 test_util.cc:268] Using random seed: 1142635159
I20240503 02:13:31.063858 3671 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54356:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:13:31.065197 3671 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:13:31.122802 3903 tablet_service.cc:1467] Processing CreateTablet for tablet 33ee6a0a5f584efe81cf64179fd6c4f0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240503 02:13:31.123333 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ee6a0a5f584efe81cf64179fd6c4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.123488 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 05f3249a7b59413792a08471c2ae4328 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240503 02:13:31.123886 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f3249a7b59413792a08471c2ae4328. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.124552 3838 tablet_service.cc:1467] Processing CreateTablet for tablet b9f36b48aefc45f88e8ad43c2a473cbe (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240503 02:13:31.124920 3838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f36b48aefc45f88e8ad43c2a473cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.125821 3830 tablet_service.cc:1467] Processing CreateTablet for tablet 2f349445bf4c4fb9a6a38825241ee2ca (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240503 02:13:31.125895 3904 tablet_service.cc:1467] Processing CreateTablet for tablet 2f349445bf4c4fb9a6a38825241ee2ca (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240503 02:13:31.126200 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f349445bf4c4fb9a6a38825241ee2ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.126157 3835 tablet_service.cc:1467] Processing CreateTablet for tablet ba261cc9d5784f15aaeeb87e8a7cae51 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240503 02:13:31.126310 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f349445bf4c4fb9a6a38825241ee2ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.126510 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba261cc9d5784f15aaeeb87e8a7cae51. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.129014 3837 tablet_service.cc:1467] Processing CreateTablet for tablet a1c6e46e6b9f47b199a76a6dc02d5a94 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240503 02:13:31.129376 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c6e46e6b9f47b199a76a6dc02d5a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.129876 3905 tablet_service.cc:1467] Processing CreateTablet for tablet d43c2f4090f14b4baaf2a55a857ed245 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240503 02:13:31.130163 3832 tablet_service.cc:1467] Processing CreateTablet for tablet e1e8091f13274e0e9fd5eb58eedb6f9c (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240503 02:13:31.130277 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43c2f4090f14b4baaf2a55a857ed245. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.130524 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1e8091f13274e0e9fd5eb58eedb6f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.133857 3906 tablet_service.cc:1467] Processing CreateTablet for tablet e1e8091f13274e0e9fd5eb58eedb6f9c (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240503 02:13:31.134142 3829 tablet_service.cc:1467] Processing CreateTablet for tablet 33ee6a0a5f584efe81cf64179fd6c4f0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240503 02:13:31.134280 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1e8091f13274e0e9fd5eb58eedb6f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.134521 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ee6a0a5f584efe81cf64179fd6c4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.133857 3831 tablet_service.cc:1467] Processing CreateTablet for tablet d43c2f4090f14b4baaf2a55a857ed245 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240503 02:13:31.136023 3831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43c2f4090f14b4baaf2a55a857ed245. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.136883 3837 tablet_service.cc:1467] Processing CreateTablet for tablet ffacc2d4cc534452ad50a87526626004 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240503 02:13:31.137311 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffacc2d4cc534452ad50a87526626004. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.137814 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 141b1d61e275407a9827ce8ba0bd1913 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240503 02:13:31.138182 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 141b1d61e275407a9827ce8ba0bd1913. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.125895 3836 tablet_service.cc:1467] Processing CreateTablet for tablet cf89d2bae6d8488c8c774557f4503365 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240503 02:13:31.141781 3908 tablet_service.cc:1467] Processing CreateTablet for tablet 05f3249a7b59413792a08471c2ae4328 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240503 02:13:31.129876 3833 tablet_service.cc:1467] Processing CreateTablet for tablet 141b1d61e275407a9827ce8ba0bd1913 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240503 02:13:31.142508 3831 tablet_service.cc:1467] Processing CreateTablet for tablet 948ee5eebd9c405fa85066b3a57ebe02 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240503 02:13:31.144479 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f3249a7b59413792a08471c2ae4328. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.146556 3832 tablet_service.cc:1467] Processing CreateTablet for tablet 1101bf1e3c82448ebf8747b8533fd2f0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240503 02:13:31.147778 3836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf89d2bae6d8488c8c774557f4503365. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.147691 3909 tablet_service.cc:1467] Processing CreateTablet for tablet ba261cc9d5784f15aaeeb87e8a7cae51 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240503 02:13:31.148917 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 141b1d61e275407a9827ce8ba0bd1913. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.149820 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1101bf1e3c82448ebf8747b8533fd2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.150302 3831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948ee5eebd9c405fa85066b3a57ebe02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.151453 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 10fb3ffa08c84ffbbd779d35a68ff7f6 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240503 02:13:31.151737 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fb3ffa08c84ffbbd779d35a68ff7f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.152057 3912 tablet_service.cc:1467] Processing CreateTablet for tablet b9f36b48aefc45f88e8ad43c2a473cbe (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240503 02:13:31.152985 3910 tablet_service.cc:1467] Processing CreateTablet for tablet cf89d2bae6d8488c8c774557f4503365 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240503 02:13:31.153030 3911 tablet_service.cc:1467] Processing CreateTablet for tablet a1c6e46e6b9f47b199a76a6dc02d5a94 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240503 02:13:31.153658 3909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba261cc9d5784f15aaeeb87e8a7cae51. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.154457 3905 tablet_service.cc:1467] Processing CreateTablet for tablet 948ee5eebd9c405fa85066b3a57ebe02 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240503 02:13:31.154793 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948ee5eebd9c405fa85066b3a57ebe02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.155071 3830 tablet_service.cc:1467] Processing CreateTablet for tablet 23b76a1a9dea45aebd7fea02fbc9cb42 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240503 02:13:31.155673 3911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c6e46e6b9f47b199a76a6dc02d5a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.156253 3903 tablet_service.cc:1467] Processing CreateTablet for tablet ffacc2d4cc534452ad50a87526626004 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240503 02:13:31.156854 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b76a1a9dea45aebd7fea02fbc9cb42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.157665 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f36b48aefc45f88e8ad43c2a473cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.159655 3837 tablet_service.cc:1467] Processing CreateTablet for tablet 0ea517f608bc44a3a34d6fa0fa37e147 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240503 02:13:31.160127 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ea517f608bc44a3a34d6fa0fa37e147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.161237 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf89d2bae6d8488c8c774557f4503365. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.161645 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffacc2d4cc534452ad50a87526626004. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.167171 3962 tablet_bootstrap.cc:492] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.176472 3904 tablet_service.cc:1467] Processing CreateTablet for tablet 1101bf1e3c82448ebf8747b8533fd2f0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240503 02:13:31.176808 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1101bf1e3c82448ebf8747b8533fd2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.177975 3908 tablet_service.cc:1467] Processing CreateTablet for tablet 10fb3ffa08c84ffbbd779d35a68ff7f6 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240503 02:13:31.178360 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fb3ffa08c84ffbbd779d35a68ff7f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.179458 3911 tablet_service.cc:1467] Processing CreateTablet for tablet 23b76a1a9dea45aebd7fea02fbc9cb42 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240503 02:13:31.179800 3911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b76a1a9dea45aebd7fea02fbc9cb42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.185331 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 0ea517f608bc44a3a34d6fa0fa37e147 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240503 02:13:31.186532 3763 tablet_service.cc:1467] Processing CreateTablet for tablet b9f36b48aefc45f88e8ad43c2a473cbe (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION VALUES < 36398027
I20240503 02:13:31.187485 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ea517f608bc44a3a34d6fa0fa37e147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.188339 3759 tablet_service.cc:1467] Processing CreateTablet for tablet 05f3249a7b59413792a08471c2ae4328 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20240503 02:13:31.189783 3762 tablet_service.cc:1467] Processing CreateTablet for tablet a1c6e46e6b9f47b199a76a6dc02d5a94 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20240503 02:13:31.190806 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 141b1d61e275407a9827ce8ba0bd1913 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20240503 02:13:31.191890 3836 tablet_service.cc:1467] Processing CreateTablet for tablet 02fb95baa11649fdbf58393d0de53fff (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240503 02:13:31.192173 3836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02fb95baa11649fdbf58393d0de53fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.195204 3761 tablet_service.cc:1467] Processing CreateTablet for tablet cf89d2bae6d8488c8c774557f4503365 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20240503 02:13:31.195461 3756 tablet_service.cc:1467] Processing CreateTablet for tablet d43c2f4090f14b4baaf2a55a857ed245 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20240503 02:13:31.198840 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f36b48aefc45f88e8ad43c2a473cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.199502 3906 tablet_service.cc:1467] Processing CreateTablet for tablet 02fb95baa11649fdbf58393d0de53fff (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240503 02:13:31.200582 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf89d2bae6d8488c8c774557f4503365. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.200809 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 141b1d61e275407a9827ce8ba0bd1913. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.195426 3760 tablet_service.cc:1467] Processing CreateTablet for tablet ba261cc9d5784f15aaeeb87e8a7cae51 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20240503 02:13:31.199151 3829 tablet_service.cc:1467] Processing CreateTablet for tablet 43cb8007da5342d29290016341c5999b (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240503 02:13:31.201232 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cb8007da5342d29290016341c5999b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.201253 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba261cc9d5784f15aaeeb87e8a7cae51. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.201650 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f3249a7b59413792a08471c2ae4328. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.200819 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02fb95baa11649fdbf58393d0de53fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.195842 3754 tablet_service.cc:1467] Processing CreateTablet for tablet 33ee6a0a5f584efe81cf64179fd6c4f0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20240503 02:13:31.194567 3963 tablet_bootstrap.cc:492] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.205057 3835 tablet_service.cc:1467] Processing CreateTablet for tablet 353eaefa93a64cd19b50fcfb916bfed6 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240503 02:13:31.195652 3755 tablet_service.cc:1467] Processing CreateTablet for tablet 2f349445bf4c4fb9a6a38825241ee2ca (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20240503 02:13:31.198734 3757 tablet_service.cc:1467] Processing CreateTablet for tablet e1e8091f13274e0e9fd5eb58eedb6f9c (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20240503 02:13:31.202633 3762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c6e46e6b9f47b199a76a6dc02d5a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.205971 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ee6a0a5f584efe81cf64179fd6c4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.206619 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43c2f4090f14b4baaf2a55a857ed245. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.207639 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353eaefa93a64cd19b50fcfb916bfed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.208876 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f349445bf4c4fb9a6a38825241ee2ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.210510 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1e8091f13274e0e9fd5eb58eedb6f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.216949 3912 tablet_service.cc:1467] Processing CreateTablet for tablet 43cb8007da5342d29290016341c5999b (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240503 02:13:31.218290 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cb8007da5342d29290016341c5999b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.218989 3962 tablet_bootstrap.cc:654] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.219002 3838 tablet_service.cc:1467] Processing CreateTablet for tablet ab19c51fbe2e4417b2fb32f20d51d5bc (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240503 02:13:31.219331 3838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab19c51fbe2e4417b2fb32f20d51d5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.222347 3905 tablet_service.cc:1467] Processing CreateTablet for tablet 353eaefa93a64cd19b50fcfb916bfed6 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240503 02:13:31.223070 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353eaefa93a64cd19b50fcfb916bfed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.225251 3907 tablet_service.cc:1467] Processing CreateTablet for tablet ab19c51fbe2e4417b2fb32f20d51d5bc (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240503 02:13:31.225659 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab19c51fbe2e4417b2fb32f20d51d5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.230716 3963 tablet_bootstrap.cc:654] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.238301 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 518cefb57371493496e276f5a10fd53d (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240503 02:13:31.238735 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518cefb57371493496e276f5a10fd53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.241233 3757 tablet_service.cc:1467] Processing CreateTablet for tablet ffacc2d4cc534452ad50a87526626004 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20240503 02:13:31.244035 3831 tablet_service.cc:1467] Processing CreateTablet for tablet 518cefb57371493496e276f5a10fd53d (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240503 02:13:31.244421 3831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518cefb57371493496e276f5a10fd53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.248039 3754 tablet_service.cc:1467] Processing CreateTablet for tablet 948ee5eebd9c405fa85066b3a57ebe02 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20240503 02:13:31.248581 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948ee5eebd9c405fa85066b3a57ebe02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.255810 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 48e4e31c162b4f5aabed85d5183c7ad5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240503 02:13:31.255995 3906 tablet_service.cc:1467] Processing CreateTablet for tablet 48e4e31c162b4f5aabed85d5183c7ad5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240503 02:13:31.256278 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e4e31c162b4f5aabed85d5183c7ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.256323 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e4e31c162b4f5aabed85d5183c7ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.264505 3755 tablet_service.cc:1467] Processing CreateTablet for tablet 1101bf1e3c82448ebf8747b8533fd2f0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20240503 02:13:31.264914 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1101bf1e3c82448ebf8747b8533fd2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.268136 3832 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1aea07837f41a281d484e37eab5744 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240503 02:13:31.268538 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1aea07837f41a281d484e37eab5744. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.269165 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffacc2d4cc534452ad50a87526626004. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.271314 3910 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1aea07837f41a281d484e37eab5744 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240503 02:13:31.271699 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1aea07837f41a281d484e37eab5744. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.271908 3905 tablet_service.cc:1467] Processing CreateTablet for tablet de9a7019777d47c48621b1b40e7ef540 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240503 02:13:31.272132 3761 tablet_service.cc:1467] Processing CreateTablet for tablet 10fb3ffa08c84ffbbd779d35a68ff7f6 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20240503 02:13:31.272388 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9a7019777d47c48621b1b40e7ef540. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.272527 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fb3ffa08c84ffbbd779d35a68ff7f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.273373 3904 tablet_service.cc:1467] Processing CreateTablet for tablet d3a88f9cd3fa40c88cc361f53c906d4e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240503 02:13:31.273622 3829 tablet_service.cc:1467] Processing CreateTablet for tablet de9a7019777d47c48621b1b40e7ef540 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240503 02:13:31.273873 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a88f9cd3fa40c88cc361f53c906d4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.274093 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9a7019777d47c48621b1b40e7ef540. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.275007 3912 tablet_service.cc:1467] Processing CreateTablet for tablet 4094a9a8c23046b6a2f27c451d8347f3 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240503 02:13:31.275063 3833 tablet_service.cc:1467] Processing CreateTablet for tablet 4094a9a8c23046b6a2f27c451d8347f3 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240503 02:13:31.283823 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4094a9a8c23046b6a2f27c451d8347f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.287551 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4094a9a8c23046b6a2f27c451d8347f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.291916 3835 tablet_service.cc:1467] Processing CreateTablet for tablet d3a88f9cd3fa40c88cc361f53c906d4e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240503 02:13:31.292443 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a88f9cd3fa40c88cc361f53c906d4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.293622 3763 tablet_service.cc:1467] Processing CreateTablet for tablet 02fb95baa11649fdbf58393d0de53fff (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20240503 02:13:31.294152 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02fb95baa11649fdbf58393d0de53fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.294003 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 0ea517f608bc44a3a34d6fa0fa37e147 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20240503 02:13:31.294521 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ea517f608bc44a3a34d6fa0fa37e147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.300045 3762 tablet_service.cc:1467] Processing CreateTablet for tablet 23b76a1a9dea45aebd7fea02fbc9cb42 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20240503 02:13:31.305678 3909 tablet_service.cc:1467] Processing CreateTablet for tablet ba1dc34edc2e42aab621736c3ae036e3 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240503 02:13:31.306105 3909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1dc34edc2e42aab621736c3ae036e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.308876 3755 tablet_service.cc:1467] Processing CreateTablet for tablet 43cb8007da5342d29290016341c5999b (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20240503 02:13:31.309579 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 73d1771c9b104e52a24a0411f05f0f94 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240503 02:13:31.309841 3762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b76a1a9dea45aebd7fea02fbc9cb42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.313575 3838 tablet_service.cc:1467] Processing CreateTablet for tablet 73d1771c9b104e52a24a0411f05f0f94 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240503 02:13:31.313956 3838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d1771c9b104e52a24a0411f05f0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.317632 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cb8007da5342d29290016341c5999b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.319140 3830 tablet_service.cc:1467] Processing CreateTablet for tablet ba1dc34edc2e42aab621736c3ae036e3 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240503 02:13:31.319834 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d1771c9b104e52a24a0411f05f0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.321593 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1dc34edc2e42aab621736c3ae036e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.322399 3837 tablet_service.cc:1467] Processing CreateTablet for tablet 89c32e8b3939447ab864df8945502369 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240503 02:13:31.324800 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 3450807dfc52460dbcbbd75dc8b71cb9 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240503 02:13:31.326025 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c32e8b3939447ab864df8945502369. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.326822 3760 tablet_service.cc:1467] Processing CreateTablet for tablet 353eaefa93a64cd19b50fcfb916bfed6 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20240503 02:13:31.328307 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3450807dfc52460dbcbbd75dc8b71cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.330113 3962 tablet_bootstrap.cc:492] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.334795 3963 tablet_bootstrap.cc:492] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.335206 3962 ts_tablet_manager.cc:1367] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.163s user 0.006s sys 0.010s
I20240503 02:13:31.335403 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353eaefa93a64cd19b50fcfb916bfed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.338920 3963 ts_tablet_manager.cc:1367] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.141s user 0.023s sys 0.049s
I20240503 02:13:31.338986 3908 tablet_service.cc:1467] Processing CreateTablet for tablet 3450807dfc52460dbcbbd75dc8b71cb9 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240503 02:13:31.340536 3911 tablet_service.cc:1467] Processing CreateTablet for tablet 89c32e8b3939447ab864df8945502369 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240503 02:13:31.344296 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3450807dfc52460dbcbbd75dc8b71cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.345989 3906 tablet_service.cc:1467] Processing CreateTablet for tablet 31d78763218f4ea3af2c4f4620e3cde4 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240503 02:13:31.346628 3756 tablet_service.cc:1467] Processing CreateTablet for tablet ab19c51fbe2e4417b2fb32f20d51d5bc (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20240503 02:13:31.348446 3911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c32e8b3939447ab864df8945502369. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.348551 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab19c51fbe2e4417b2fb32f20d51d5bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.350065 3964 tablet_bootstrap.cc:492] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.351020 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31d78763218f4ea3af2c4f4620e3cde4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.351649 3836 tablet_service.cc:1467] Processing CreateTablet for tablet 31d78763218f4ea3af2c4f4620e3cde4 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240503 02:13:31.353632 3836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31d78763218f4ea3af2c4f4620e3cde4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.354454 3903 tablet_service.cc:1467] Processing CreateTablet for tablet 5127e67d5abf450eafbf14b22cde7939 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240503 02:13:31.355036 3962 raft_consensus.cc:348] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.356243 3962 raft_consensus.cc:374] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.358222 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5127e67d5abf450eafbf14b22cde7939. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.364315 3962 raft_consensus.cc:729] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.366385 3963 raft_consensus.cc:348] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.366039 3962 consensus_queue.cc:260] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.368960 3963 raft_consensus.cc:374] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.378844 3832 tablet_service.cc:1467] Processing CreateTablet for tablet 5127e67d5abf450eafbf14b22cde7939 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240503 02:13:31.381191 3873 heartbeater.cc:503] Master 127.2.157.126:40759 was elected leader, sending a full tablet report...
I20240503 02:13:31.381829 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5127e67d5abf450eafbf14b22cde7939. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.382247 3963 raft_consensus.cc:729] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.384289 3962 ts_tablet_manager.cc:1398] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.040s user 0.001s sys 0.008s
I20240503 02:13:31.386718 3963 consensus_queue.cc:260] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.392544 3962 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.394054 3763 tablet_service.cc:1467] Processing CreateTablet for tablet 518cefb57371493496e276f5a10fd53d (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20240503 02:13:31.398209 3909 tablet_service.cc:1467] Processing CreateTablet for tablet 34af2cfd01c7406da5ed8da1ce0cf129 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240503 02:13:31.398164 3761 tablet_service.cc:1467] Processing CreateTablet for tablet 48e4e31c162b4f5aabed85d5183c7ad5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20240503 02:13:31.405387 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518cefb57371493496e276f5a10fd53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.405676 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e4e31c162b4f5aabed85d5183c7ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.410598 3909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34af2cfd01c7406da5ed8da1ce0cf129. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.420220 3910 tablet_service.cc:1467] Processing CreateTablet for tablet 965bbbd3d2e74bdc81d09b292e5ed8c5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240503 02:13:31.429493 3835 tablet_service.cc:1467] Processing CreateTablet for tablet 34af2cfd01c7406da5ed8da1ce0cf129 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240503 02:13:31.430421 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965bbbd3d2e74bdc81d09b292e5ed8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.434372 3830 tablet_service.cc:1467] Processing CreateTablet for tablet 965bbbd3d2e74bdc81d09b292e5ed8c5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240503 02:13:31.435029 3759 tablet_service.cc:1467] Processing CreateTablet for tablet 4094a9a8c23046b6a2f27c451d8347f3 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20240503 02:13:31.435351 3833 tablet_service.cc:1467] Processing CreateTablet for tablet 89de6eace7d643e2a78385a67c771991 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240503 02:13:31.435647 3967 raft_consensus.cc:482] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.436605 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4094a9a8c23046b6a2f27c451d8347f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.436290 3967 raft_consensus.cc:504] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.438037 3969 raft_consensus.cc:482] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.435604 3904 tablet_service.cc:1467] Processing CreateTablet for tablet 89de6eace7d643e2a78385a67c771991 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240503 02:13:31.439791 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34af2cfd01c7406da5ed8da1ce0cf129. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.442023 3757 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1aea07837f41a281d484e37eab5744 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20240503 02:13:31.442420 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1aea07837f41a281d484e37eab5744. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.447412 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89de6eace7d643e2a78385a67c771991. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.447619 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89de6eace7d643e2a78385a67c771991. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.449100 3964 tablet_bootstrap.cc:654] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.451165 3967 leader_election.cc:288] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:31.453089 3962 tablet_bootstrap.cc:654] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.454723 3912 tablet_service.cc:1467] Processing CreateTablet for tablet b04a17e29c044fcaab0eb6e836eca141 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240503 02:13:31.457772 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965bbbd3d2e74bdc81d09b292e5ed8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.461019 3963 ts_tablet_manager.cc:1398] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.107s user 0.019s sys 0.019s
I20240503 02:13:31.461309 3947 heartbeater.cc:503] Master 127.2.157.126:40759 was elected leader, sending a full tablet report...
I20240503 02:13:31.463265 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04a17e29c044fcaab0eb6e836eca141. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.469400 3963 tablet_bootstrap.cc:492] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.476820 3838 tablet_service.cc:1467] Processing CreateTablet for tablet a5e9aab321bd48c38b354316119370d2 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240503 02:13:31.476794 3758 tablet_service.cc:1467] Processing CreateTablet for tablet de9a7019777d47c48621b1b40e7ef540 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20240503 02:13:31.477294 3760 tablet_service.cc:1467] Processing CreateTablet for tablet 73d1771c9b104e52a24a0411f05f0f94 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20240503 02:13:31.477556 3831 tablet_service.cc:1467] Processing CreateTablet for tablet b04a17e29c044fcaab0eb6e836eca141 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240503 02:13:31.479110 3907 tablet_service.cc:1467] Processing CreateTablet for tablet a5e9aab321bd48c38b354316119370d2 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240503 02:13:31.479420 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e9aab321bd48c38b354316119370d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.450461 3969 raft_consensus.cc:504] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.480705 3969 leader_election.cc:288] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:31.481966 3838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e9aab321bd48c38b354316119370d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.485852 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9a7019777d47c48621b1b40e7ef540. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.486006 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d1771c9b104e52a24a0411f05f0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.486676 3762 tablet_service.cc:1467] Processing CreateTablet for tablet d3a88f9cd3fa40c88cc361f53c906d4e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20240503 02:13:31.485774 3829 tablet_service.cc:1467] Processing CreateTablet for tablet 15eeedfc81114776a8614759c7741de9 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240503 02:13:31.488523 3754 tablet_service.cc:1467] Processing CreateTablet for tablet ba1dc34edc2e42aab621736c3ae036e3 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20240503 02:13:31.495855 3905 tablet_service.cc:1467] Processing CreateTablet for tablet 15eeedfc81114776a8614759c7741de9 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240503 02:13:31.496331 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15eeedfc81114776a8614759c7741de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.497900 3831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04a17e29c044fcaab0eb6e836eca141. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:13:31.501878 3874 tablet.cc:2369] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 02:13:31.502967 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15eeedfc81114776a8614759c7741de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.503017 3762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a88f9cd3fa40c88cc361f53c906d4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.503528 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1dc34edc2e42aab621736c3ae036e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.506152 3836 tablet_service.cc:1467] Processing CreateTablet for tablet d9da09c39261443bbf898a390937ffb4 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240503 02:13:31.506613 3836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9da09c39261443bbf898a390937ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.518860 3962 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.519023 3962 ts_tablet_manager.cc:1367] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.024s
I20240503 02:13:31.522007 3759 tablet_service.cc:1467] Processing CreateTablet for tablet 3450807dfc52460dbcbbd75dc8b71cb9 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20240503 02:13:31.522727 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3450807dfc52460dbcbbd75dc8b71cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.523667 3962 raft_consensus.cc:348] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.523859 3962 raft_consensus.cc:374] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.523908 3962 raft_consensus.cc:729] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.524077 3962 consensus_queue.cc:260] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.527351 3756 tablet_service.cc:1467] Processing CreateTablet for tablet 89c32e8b3939447ab864df8945502369 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20240503 02:13:31.527690 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c32e8b3939447ab864df8945502369. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.529711 3962 ts_tablet_manager.cc:1398] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20240503 02:13:31.530009 3962 tablet_bootstrap.cc:492] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.532313 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 01d0bfa5dbdc44928bed580d8198659c (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240503 02:13:31.532662 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d0bfa5dbdc44928bed580d8198659c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.537459 3908 tablet_service.cc:1467] Processing CreateTablet for tablet 01d0bfa5dbdc44928bed580d8198659c (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240503 02:13:31.537874 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d0bfa5dbdc44928bed580d8198659c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.538405 3903 tablet_service.cc:1467] Processing CreateTablet for tablet d9da09c39261443bbf898a390937ffb4 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240503 02:13:31.538736 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9da09c39261443bbf898a390937ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.540819 3964 tablet_bootstrap.cc:492] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.542215 3964 ts_tablet_manager.cc:1367] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.214s user 0.005s sys 0.018s
I20240503 02:13:31.544751 3837 tablet_service.cc:1467] Processing CreateTablet for tablet e8d23a9965894fc2aed70be32e716613 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240503 02:13:31.545073 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d23a9965894fc2aed70be32e716613. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.548147 3755 tablet_service.cc:1467] Processing CreateTablet for tablet 31d78763218f4ea3af2c4f4620e3cde4 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20240503 02:13:31.548420 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31d78763218f4ea3af2c4f4620e3cde4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.553925 3912 tablet_service.cc:1467] Processing CreateTablet for tablet e8d23a9965894fc2aed70be32e716613 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240503 02:13:31.554261 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d23a9965894fc2aed70be32e716613. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.559474 3911 tablet_service.cc:1467] Processing CreateTablet for tablet de6622d0a3f54d418d588fe0281d9324 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240503 02:13:31.560945 3911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6622d0a3f54d418d588fe0281d9324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.562288 3831 tablet_service.cc:1467] Processing CreateTablet for tablet de6622d0a3f54d418d588fe0281d9324 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240503 02:13:31.559734 3964 raft_consensus.cc:348] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.563462 3964 raft_consensus.cc:374] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.563547 3964 raft_consensus.cc:729] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.565759 3963 tablet_bootstrap.cc:654] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.566579 3831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6622d0a3f54d418d588fe0281d9324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.570488 3909 tablet_service.cc:1467] Processing CreateTablet for tablet f6ff048917b34a4b9cd09dee267fc078 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240503 02:13:31.570912 3909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6ff048917b34a4b9cd09dee267fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.571486 3833 tablet_service.cc:1467] Processing CreateTablet for tablet f6ff048917b34a4b9cd09dee267fc078 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240503 02:13:31.571715 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 5127e67d5abf450eafbf14b22cde7939 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20240503 02:13:31.571830 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6ff048917b34a4b9cd09dee267fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.572036 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5127e67d5abf450eafbf14b22cde7939. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.572603 3835 tablet_service.cc:1467] Processing CreateTablet for tablet ea39b9e4e11247ab95cc690e60b22b11 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240503 02:13:31.572885 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea39b9e4e11247ab95cc690e60b22b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.576210 3906 tablet_service.cc:1467] Processing CreateTablet for tablet ea39b9e4e11247ab95cc690e60b22b11 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240503 02:13:31.576496 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea39b9e4e11247ab95cc690e60b22b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.565321 3964 consensus_queue.cc:260] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.579975 3838 tablet_service.cc:1467] Processing CreateTablet for tablet 88bc3ed40ce7448a98251d25e2a09798 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240503 02:13:31.580253 3838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bc3ed40ce7448a98251d25e2a09798. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.585464 3962 tablet_bootstrap.cc:654] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.583732 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f349445bf4c4fb9a6a38825241ee2ca" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:31.590364 3812 leader_election.cc:341] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.603983 3763 tablet_service.cc:1467] Processing CreateTablet for tablet 34af2cfd01c7406da5ed8da1ce0cf129 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20240503 02:13:31.604403 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34af2cfd01c7406da5ed8da1ce0cf129. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.611709 3963 tablet_bootstrap.cc:492] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.611900 3963 ts_tablet_manager.cc:1367] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.151s user 0.005s sys 0.014s
I20240503 02:13:31.615135 3910 tablet_service.cc:1467] Processing CreateTablet for tablet 88bc3ed40ce7448a98251d25e2a09798 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240503 02:13:31.616891 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bc3ed40ce7448a98251d25e2a09798. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.625046 3832 tablet_service.cc:1467] Processing CreateTablet for tablet 793fe86859754f3d90919e150e2f3693 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240503 02:13:31.625511 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793fe86859754f3d90919e150e2f3693. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.629945 3964 ts_tablet_manager.cc:1398] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.088s user 0.016s sys 0.019s
I20240503 02:13:31.630241 3964 tablet_bootstrap.cc:492] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.630283 3798 heartbeater.cc:503] Master 127.2.157.126:40759 was elected leader, sending a full tablet report...
I20240503 02:13:31.641773 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 793fe86859754f3d90919e150e2f3693 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240503 02:13:31.642968 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793fe86859754f3d90919e150e2f3693. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.643640 3837 tablet_service.cc:1467] Processing CreateTablet for tablet 2758e6b755a9470e872f11fe2fb9001a (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240503 02:13:31.648185 3757 tablet_service.cc:1467] Processing CreateTablet for tablet 965bbbd3d2e74bdc81d09b292e5ed8c5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20240503 02:13:31.651831 3905 tablet_service.cc:1467] Processing CreateTablet for tablet 2758e6b755a9470e872f11fe2fb9001a (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240503 02:13:31.652304 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2758e6b755a9470e872f11fe2fb9001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.654196 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965bbbd3d2e74bdc81d09b292e5ed8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.654413 3829 tablet_service.cc:1467] Processing CreateTablet for tablet df08ada758d5442cba4f7bd85b445352 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240503 02:13:31.654982 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df08ada758d5442cba4f7bd85b445352. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.655347 3761 tablet_service.cc:1467] Processing CreateTablet for tablet 89de6eace7d643e2a78385a67c771991 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20240503 02:13:31.651436 3830 tablet_service.cc:1467] Processing CreateTablet for tablet ddd36fffa3034c6cbe511f4f39b078ff (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240503 02:13:31.655822 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd36fffa3034c6cbe511f4f39b078ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.656913 3963 raft_consensus.cc:348] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.658313 3832 tablet_service.cc:1467] Processing CreateTablet for tablet 942fa19a821249e1b11d625a6f712368 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240503 02:13:31.658635 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 942fa19a821249e1b11d625a6f712368. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.657230 3963 raft_consensus.cc:374] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.661175 3963 raft_consensus.cc:729] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.661592 3963 consensus_queue.cc:260] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.664731 3910 tablet_service.cc:1467] Processing CreateTablet for tablet ddd36fffa3034c6cbe511f4f39b078ff (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240503 02:13:31.665135 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd36fffa3034c6cbe511f4f39b078ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.666358 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2758e6b755a9470e872f11fe2fb9001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.667356 3912 tablet_service.cc:1467] Processing CreateTablet for tablet 942fa19a821249e1b11d625a6f712368 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240503 02:13:31.667716 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 942fa19a821249e1b11d625a6f712368. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.669268 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89de6eace7d643e2a78385a67c771991. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.673902 3963 ts_tablet_manager.cc:1398] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.062s user 0.003s sys 0.000s
I20240503 02:13:31.674547 3963 tablet_bootstrap.cc:492] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.675218 3832 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc392ed1bce416c8e0d47af8c9787bf (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240503 02:13:31.675518 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc392ed1bce416c8e0d47af8c9787bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.677723 3904 tablet_service.cc:1467] Processing CreateTablet for tablet df08ada758d5442cba4f7bd85b445352 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240503 02:13:31.678212 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df08ada758d5442cba4f7bd85b445352. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.678472 3760 tablet_service.cc:1467] Processing CreateTablet for tablet b04a17e29c044fcaab0eb6e836eca141 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20240503 02:13:31.678548 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f349445bf4c4fb9a6a38825241ee2ca" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:31.681717 3758 tablet_service.cc:1467] Processing CreateTablet for tablet a5e9aab321bd48c38b354316119370d2 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20240503 02:13:31.682024 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e9aab321bd48c38b354316119370d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.682184 3907 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc392ed1bce416c8e0d47af8c9787bf (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240503 02:13:31.682525 3833 tablet_service.cc:1467] Processing CreateTablet for tablet 072c60ef4387498abec90d6716f856b8 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240503 02:13:31.682860 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072c60ef4387498abec90d6716f856b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.682533 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc392ed1bce416c8e0d47af8c9787bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.684624 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04a17e29c044fcaab0eb6e836eca141. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.685076 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141b1d61e275407a9827ce8ba0bd1913" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:31.685580 3836 tablet_service.cc:1467] Processing CreateTablet for tablet eef59225fda142b99782f8e09d65cb9e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240503 02:13:31.685927 3836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef59225fda142b99782f8e09d65cb9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:13:31.686370 3885 leader_election.cc:341] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:31.686370 3814 leader_election.cc:341] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.686758 3756 tablet_service.cc:1467] Processing CreateTablet for tablet 15eeedfc81114776a8614759c7741de9 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20240503 02:13:31.686878 3814 leader_election.cc:302] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:31.687753 3763 tablet_service.cc:1467] Processing CreateTablet for tablet ea39b9e4e11247ab95cc690e60b22b11 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20240503 02:13:31.687855 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15eeedfc81114776a8614759c7741de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.688100 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea39b9e4e11247ab95cc690e60b22b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.689625 3967 raft_consensus.cc:2726] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:31.691632 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141b1d61e275407a9827ce8ba0bd1913" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:31.692004 3886 leader_election.cc:341] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.692045 3908 tablet_service.cc:1467] Processing CreateTablet for tablet eef59225fda142b99782f8e09d65cb9e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240503 02:13:31.692127 3886 leader_election.cc:302] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:31.692325 3969 raft_consensus.cc:2726] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:31.692364 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef59225fda142b99782f8e09d65cb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.695614 3964 tablet_bootstrap.cc:654] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.702093 3757 tablet_service.cc:1467] Processing CreateTablet for tablet d9da09c39261443bbf898a390937ffb4 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20240503 02:13:31.702504 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9da09c39261443bbf898a390937ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.703240 3962 tablet_bootstrap.cc:492] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.703500 3962 ts_tablet_manager.cc:1367] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.174s user 0.031s sys 0.024s
I20240503 02:13:31.704634 3962 raft_consensus.cc:348] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.704962 3962 raft_consensus.cc:374] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.705083 3962 raft_consensus.cc:729] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.705475 3962 consensus_queue.cc:260] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.706593 3962 ts_tablet_manager.cc:1398] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:31.706831 3962 tablet_bootstrap.cc:492] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.707365 3759 tablet_service.cc:1467] Processing CreateTablet for tablet 01d0bfa5dbdc44928bed580d8198659c (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20240503 02:13:31.707726 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d0bfa5dbdc44928bed580d8198659c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.707881 3962 tablet_bootstrap.cc:654] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.712098 3754 tablet_service.cc:1467] Processing CreateTablet for tablet e8d23a9965894fc2aed70be32e716613 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20240503 02:13:31.712453 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d23a9965894fc2aed70be32e716613. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.713941 3963 tablet_bootstrap.cc:654] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.715901 3762 tablet_service.cc:1467] Processing CreateTablet for tablet de6622d0a3f54d418d588fe0281d9324 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20240503 02:13:31.716241 3762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6622d0a3f54d418d588fe0281d9324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.716468 3755 tablet_service.cc:1467] Processing CreateTablet for tablet f6ff048917b34a4b9cd09dee267fc078 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20240503 02:13:31.716797 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6ff048917b34a4b9cd09dee267fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.723115 3837 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4972943b854b28a0c927ce1a0511d7 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240503 02:13:31.723469 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4972943b854b28a0c927ce1a0511d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.731508 3833 tablet_service.cc:1467] Processing CreateTablet for tablet a511b3bba2024783a999d7c7b616100e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240503 02:13:31.731928 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a511b3bba2024783a999d7c7b616100e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.739295 3905 tablet_service.cc:1467] Processing CreateTablet for tablet 072c60ef4387498abec90d6716f856b8 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240503 02:13:31.739686 3905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072c60ef4387498abec90d6716f856b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.740015 3906 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4972943b854b28a0c927ce1a0511d7 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240503 02:13:31.740378 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4972943b854b28a0c927ce1a0511d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.743448 3903 tablet_service.cc:1467] Processing CreateTablet for tablet a511b3bba2024783a999d7c7b616100e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240503 02:13:31.743842 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a511b3bba2024783a999d7c7b616100e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.748319 3834 tablet_service.cc:1467] Processing CreateTablet for tablet 86a25012564547caaa8289902be2b42f (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240503 02:13:31.748682 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a25012564547caaa8289902be2b42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.751883 3911 tablet_service.cc:1467] Processing CreateTablet for tablet 86a25012564547caaa8289902be2b42f (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240503 02:13:31.752861 3911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a25012564547caaa8289902be2b42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.754197 3909 tablet_service.cc:1467] Processing CreateTablet for tablet 05ae769e64a54f46b68f4699ec8ffd05 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240503 02:13:31.754513 3909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ae769e64a54f46b68f4699ec8ffd05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.754285 3964 tablet_bootstrap.cc:492] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.754810 3964 ts_tablet_manager.cc:1367] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.125s user 0.004s sys 0.000s
I20240503 02:13:31.754889 3761 tablet_service.cc:1467] Processing CreateTablet for tablet 88bc3ed40ce7448a98251d25e2a09798 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20240503 02:13:31.755218 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bc3ed40ce7448a98251d25e2a09798. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.755646 3964 raft_consensus.cc:348] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.755846 3964 raft_consensus.cc:374] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.755908 3964 raft_consensus.cc:729] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.756098 3964 consensus_queue.cc:260] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.756194 3759 tablet_service.cc:1467] Processing CreateTablet for tablet 793fe86859754f3d90919e150e2f3693 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20240503 02:13:31.756495 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793fe86859754f3d90919e150e2f3693. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.753839 3835 tablet_service.cc:1467] Processing CreateTablet for tablet e8fcb19cccf64822aa8ba1dd55bfc5a5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240503 02:13:31.754467 3910 tablet_service.cc:1467] Processing CreateTablet for tablet e8fcb19cccf64822aa8ba1dd55bfc5a5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240503 02:13:31.757326 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8fcb19cccf64822aa8ba1dd55bfc5a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.757402 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8fcb19cccf64822aa8ba1dd55bfc5a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.754601 3829 tablet_service.cc:1467] Processing CreateTablet for tablet 05ae769e64a54f46b68f4699ec8ffd05 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240503 02:13:31.759148 3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ae769e64a54f46b68f4699ec8ffd05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.760010 3763 tablet_service.cc:1467] Processing CreateTablet for tablet 2758e6b755a9470e872f11fe2fb9001a (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20240503 02:13:31.760485 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2758e6b755a9470e872f11fe2fb9001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.759109 3837 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8c6d907cff43638bf43b45e1c3edd0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240503 02:13:31.761474 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8c6d907cff43638bf43b45e1c3edd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.762643 3962 tablet_bootstrap.cc:492] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.762763 3962 ts_tablet_manager.cc:1367] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.033s
I20240503 02:13:31.763381 3964 ts_tablet_manager.cc:1398] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20240503 02:13:31.763624 3964 tablet_bootstrap.cc:492] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.761336 3908 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8c6d907cff43638bf43b45e1c3edd0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240503 02:13:31.765230 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8c6d907cff43638bf43b45e1c3edd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.771505 3964 tablet_bootstrap.cc:654] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.768648 3757 tablet_service.cc:1467] Processing CreateTablet for tablet ddd36fffa3034c6cbe511f4f39b078ff (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20240503 02:13:31.775190 3757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddd36fffa3034c6cbe511f4f39b078ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.778980 3963 tablet_bootstrap.cc:492] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.779101 3963 ts_tablet_manager.cc:1367] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.105s user 0.030s sys 0.004s
I20240503 02:13:31.784291 3754 tablet_service.cc:1467] Processing CreateTablet for tablet 072c60ef4387498abec90d6716f856b8 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20240503 02:13:31.784690 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072c60ef4387498abec90d6716f856b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.787539 3962 raft_consensus.cc:348] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.787783 3962 raft_consensus.cc:374] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.787855 3962 raft_consensus.cc:729] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.788118 3962 consensus_queue.cc:260] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.789743 3756 tablet_service.cc:1467] Processing CreateTablet for tablet df08ada758d5442cba4f7bd85b445352 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20240503 02:13:31.790078 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df08ada758d5442cba4f7bd85b445352. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.790796 3967 raft_consensus.cc:482] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.790910 3967 raft_consensus.cc:504] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.791530 3967 leader_election.cc:288] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:31.791983 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1e8091f13274e0e9fd5eb58eedb6f9c" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:31.792951 3814 leader_election.cc:341] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.793427 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1e8091f13274e0e9fd5eb58eedb6f9c" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:31.793615 3773 raft_consensus.cc:2454] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:31.794191 3812 leader_election.cc:302] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:31.794800 3967 raft_consensus.cc:2781] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:31.794867 3967 raft_consensus.cc:482] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:31.794929 3967 raft_consensus.cc:3037] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:31.797353 3967 raft_consensus.cc:504] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.797995 3755 tablet_service.cc:1467] Processing CreateTablet for tablet 942fa19a821249e1b11d625a6f712368 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20240503 02:13:31.798873 3962 ts_tablet_manager.cc:1398] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.036s user 0.003s sys 0.000s
I20240503 02:13:31.799556 3962 tablet_bootstrap.cc:492] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.794209 3762 tablet_service.cc:1467] Processing CreateTablet for tablet 4bc392ed1bce416c8e0d47af8c9787bf (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20240503 02:13:31.802210 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 942fa19a821249e1b11d625a6f712368. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.802366 3762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bc392ed1bce416c8e0d47af8c9787bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.805418 3975 raft_consensus.cc:482] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.805596 3975 raft_consensus.cc:504] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.806401 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1e8091f13274e0e9fd5eb58eedb6f9c" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:31.806528 3773 raft_consensus.cc:3037] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:31.808449 3754 tablet_service.cc:1467] Processing CreateTablet for tablet eef59225fda142b99782f8e09d65cb9e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20240503 02:13:31.808888 3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef59225fda142b99782f8e09d65cb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.809787 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4972943b854b28a0c927ce1a0511d7 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20240503 02:13:31.810182 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4972943b854b28a0c927ce1a0511d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.814121 3975 leader_election.cc:288] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:31.815469 3967 leader_election.cc:288] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:31.817471 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1e8091f13274e0e9fd5eb58eedb6f9c" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
W20240503 02:13:31.817976 3814 leader_election.cc:341] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.818852 3759 tablet_service.cc:1467] Processing CreateTablet for tablet a511b3bba2024783a999d7c7b616100e (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20240503 02:13:31.819162 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a511b3bba2024783a999d7c7b616100e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.828334 3964 tablet_bootstrap.cc:492] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.828464 3964 ts_tablet_manager.cc:1367] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.065s user 0.021s sys 0.019s
I20240503 02:13:31.832039 3963 raft_consensus.cc:348] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.832288 3963 raft_consensus.cc:374] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.832360 3963 raft_consensus.cc:729] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.834618 3963 consensus_queue.cc:260] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.839501 3963 ts_tablet_manager.cc:1398] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.060s user 0.001s sys 0.001s
I20240503 02:13:31.841693 3963 tablet_bootstrap.cc:492] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.850622 3964 raft_consensus.cc:348] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.850898 3964 raft_consensus.cc:374] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.850971 3964 raft_consensus.cc:729] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.851231 3964 consensus_queue.cc:260] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.854866 3761 tablet_service.cc:1467] Processing CreateTablet for tablet 86a25012564547caaa8289902be2b42f (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20240503 02:13:31.855410 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a25012564547caaa8289902be2b42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.858073 3773 raft_consensus.cc:2454] T e1e8091f13274e0e9fd5eb58eedb6f9c P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:31.863804 3760 tablet_service.cc:1467] Processing CreateTablet for tablet e8fcb19cccf64822aa8ba1dd55bfc5a5 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20240503 02:13:31.864338 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8fcb19cccf64822aa8ba1dd55bfc5a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.868146 3759 tablet_service.cc:1467] Processing CreateTablet for tablet 05ae769e64a54f46b68f4699ec8ffd05 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20240503 02:13:31.868602 3759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ae769e64a54f46b68f4699ec8ffd05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.870759 3964 ts_tablet_manager.cc:1398] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.042s user 0.001s sys 0.002s
I20240503 02:13:31.871670 3969 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.871824 3969 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.872684 3812 leader_election.cc:302] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:31.872941 3967 raft_consensus.cc:2781] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:31.873340 3969 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:31.873648 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:31.874039 3886 leader_election.cc:341] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.874277 3964 tablet_bootstrap.cc:492] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.876425 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:31.876574 3848 raft_consensus.cc:2454] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:31.876863 3885 leader_election.cc:302] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:31.879376 3969 raft_consensus.cc:2781] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:31.879439 3969 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:31.879482 3969 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:31.881368 3719 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 02:13:31.882755 3756 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8c6d907cff43638bf43b45e1c3edd0 (DEFAULT_TABLE table=test-workload [id=c49adf327c244a9aa46bc38493fc2805]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20240503 02:13:31.883126 3967 raft_consensus.cc:686] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:31.883652 3962 tablet_bootstrap.cc:654] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.883788 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8c6d907cff43638bf43b45e1c3edd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:31.883783 3967 consensus_queue.cc:237] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.890587 3969 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.890847 3963 tablet_bootstrap.cc:654] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.891810 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:31.892683 3969 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
W20240503 02:13:31.892823 3886 leader_election.cc:341] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.894296 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:31.894495 3848 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:31.894742 3984 raft_consensus.cc:482] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.894868 3984 raft_consensus.cc:504] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.899434 3962 tablet_bootstrap.cc:492] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.899564 3962 ts_tablet_manager.cc:1367] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.100s user 0.005s sys 0.005s
I20240503 02:13:31.899595 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c6e46e6b9f47b199a76a6dc02d5a94" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:31.899600 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c6e46e6b9f47b199a76a6dc02d5a94" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:31.900061 3814 leader_election.cc:341] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.900118 3984 leader_election.cc:288] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
W20240503 02:13:31.900192 3812 leader_election.cc:341] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.900269 3812 leader_election.cc:302] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:31.901424 3964 tablet_bootstrap.cc:654] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.907531 3984 raft_consensus.cc:2726] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:31.907805 3962 raft_consensus.cc:348] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.908511 3962 raft_consensus.cc:374] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.908581 3962 raft_consensus.cc:729] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.909346 3848 raft_consensus.cc:2454] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:31.909732 3885 leader_election.cc:302] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:31.909960 3969 raft_consensus.cc:2781] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:31.910086 3969 raft_consensus.cc:686] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:31.908792 3962 consensus_queue.cc:260] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.910684 3969 consensus_queue.cc:237] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.912678 3962 ts_tablet_manager.cc:1398] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.013s user 0.001s sys 0.002s
I20240503 02:13:31.912928 3962 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.912178 3669 catalog_manager.cc:5484] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:31.923650 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
W20240503 02:13:31.924136 3736 leader_election.cc:341] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.926323 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:31.926846 3739 leader_election.cc:341] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.926964 3739 leader_election.cc:302] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:31.927186 3975 raft_consensus.cc:2726] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:31.930191 3963 tablet_bootstrap.cc:492] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.931324 3963 ts_tablet_manager.cc:1367] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.090s user 0.012s sys 0.001s
I20240503 02:13:31.930650 3962 tablet_bootstrap.cc:654] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.935312 3963 raft_consensus.cc:348] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.935556 3963 raft_consensus.cc:374] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.935621 3963 raft_consensus.cc:729] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.935813 3963 consensus_queue.cc:260] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.937556 3963 ts_tablet_manager.cc:1398] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20240503 02:13:31.937847 3963 tablet_bootstrap.cc:492] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.939610 3670 catalog_manager.cc:5484] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:31.940315 3964 tablet_bootstrap.cc:492] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.940418 3964 ts_tablet_manager.cc:1367] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.069s user 0.005s sys 0.005s
I20240503 02:13:31.941051 3964 raft_consensus.cc:348] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.941201 3964 raft_consensus.cc:374] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.941260 3964 raft_consensus.cc:729] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.943516 3962 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.943619 3962 ts_tablet_manager.cc:1367] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.002s
I20240503 02:13:31.944036 3962 raft_consensus.cc:348] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.944206 3962 raft_consensus.cc:374] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.944329 3962 raft_consensus.cc:729] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.944540 3962 consensus_queue.cc:260] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.944206 3964 consensus_queue.cc:260] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.944923 3962 ts_tablet_manager.cc:1398] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:13:31.945025 3964 ts_tablet_manager.cc:1398] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240503 02:13:31.945158 3962 tablet_bootstrap.cc:492] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.946611 3962 tablet_bootstrap.cc:654] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.944149 3963 tablet_bootstrap.cc:654] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.947492 3964 tablet_bootstrap.cc:492] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.949146 3964 tablet_bootstrap.cc:654] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.949586 3962 tablet_bootstrap.cc:492] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.949733 3962 ts_tablet_manager.cc:1367] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240503 02:13:31.950541 3962 raft_consensus.cc:348] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.950767 3962 raft_consensus.cc:374] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.950831 3962 raft_consensus.cc:729] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.951095 3962 consensus_queue.cc:260] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.951812 3962 ts_tablet_manager.cc:1398] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:31.952101 3962 tablet_bootstrap.cc:492] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.952543 3963 tablet_bootstrap.cc:492] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.952697 3963 ts_tablet_manager.cc:1367] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20240503 02:13:31.953403 3963 raft_consensus.cc:348] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.953670 3963 raft_consensus.cc:374] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.953784 3963 raft_consensus.cc:729] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.954052 3962 tablet_bootstrap.cc:654] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.954070 3963 consensus_queue.cc:260] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.954784 3963 ts_tablet_manager.cc:1398] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:31.955070 3963 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.956611 3963 tablet_bootstrap.cc:654] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.957214 3964 tablet_bootstrap.cc:492] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.957331 3964 ts_tablet_manager.cc:1367] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20240503 02:13:31.958066 3964 raft_consensus.cc:348] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.958199 3964 raft_consensus.cc:374] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.958253 3964 raft_consensus.cc:729] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.958429 3964 consensus_queue.cc:260] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.958498 3963 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.958684 3963 ts_tablet_manager.cc:1367] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:13:31.958879 3964 ts_tablet_manager.cc:1398] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240503 02:13:31.959093 3964 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.959312 3963 raft_consensus.cc:348] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.959471 3963 raft_consensus.cc:374] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.959576 3963 raft_consensus.cc:729] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.959821 3963 consensus_queue.cc:260] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.960332 3964 tablet_bootstrap.cc:654] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.960359 3963 ts_tablet_manager.cc:1398] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:31.960610 3963 tablet_bootstrap.cc:492] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.961874 3962 tablet_bootstrap.cc:492] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.961993 3962 ts_tablet_manager.cc:1367] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20240503 02:13:31.962098 3964 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.962206 3963 tablet_bootstrap.cc:654] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.962205 3964 ts_tablet_manager.cc:1367] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240503 02:13:31.962618 3962 raft_consensus.cc:348] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.962770 3962 raft_consensus.cc:374] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.962839 3962 raft_consensus.cc:729] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.962774 3964 raft_consensus.cc:348] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.962973 3964 raft_consensus.cc:374] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.963043 3964 raft_consensus.cc:729] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.963006 3962 consensus_queue.cc:260] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.963264 3964 consensus_queue.cc:260] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.963445 3962 ts_tablet_manager.cc:1398] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:13:31.963652 3962 tablet_bootstrap.cc:492] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.965097 3962 tablet_bootstrap.cc:654] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.965657 3964 ts_tablet_manager.cc:1398] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:31.965874 3964 tablet_bootstrap.cc:492] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.967576 3964 tablet_bootstrap.cc:654] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.973726 3963 tablet_bootstrap.cc:492] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.973845 3963 ts_tablet_manager.cc:1367] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.002s
I20240503 02:13:31.974347 3962 tablet_bootstrap.cc:492] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.974344 3963 raft_consensus.cc:348] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.974475 3963 raft_consensus.cc:374] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.974527 3963 raft_consensus.cc:729] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.974529 3962 ts_tablet_manager.cc:1367] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20240503 02:13:31.974726 3963 consensus_queue.cc:260] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.975219 3963 ts_tablet_manager.cc:1398] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:13:31.975292 3962 raft_consensus.cc:348] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.975448 3963 tablet_bootstrap.cc:492] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.975451 3962 raft_consensus.cc:374] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.975600 3962 raft_consensus.cc:729] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.975721 3964 tablet_bootstrap.cc:492] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.975805 3962 consensus_queue.cc:260] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.976258 3962 ts_tablet_manager.cc:1398] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:31.975816 3964 ts_tablet_manager.cc:1367] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20240503 02:13:31.977008 3963 tablet_bootstrap.cc:654] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.976956 3964 raft_consensus.cc:348] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.977123 3964 raft_consensus.cc:374] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.977183 3964 raft_consensus.cc:729] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.977371 3964 consensus_queue.cc:260] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.977855 3964 ts_tablet_manager.cc:1398] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:13:31.978096 3964 tablet_bootstrap.cc:492] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.979568 3964 tablet_bootstrap.cc:654] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.980728 3962 tablet_bootstrap.cc:492] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:31.982968 3967 raft_consensus.cc:482] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.983270 3962 tablet_bootstrap.cc:654] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.983134 3967 raft_consensus.cc:504] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.984045 3967 leader_election.cc:288] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:31.984720 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf89d2bae6d8488c8c774557f4503365" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:31.984869 3963 tablet_bootstrap.cc:492] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:31.984982 3963 ts_tablet_manager.cc:1367] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
W20240503 02:13:31.985311 3814 leader_election.cc:341] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.985522 3963 raft_consensus.cc:348] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:31.985805 3963 raft_consensus.cc:374] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.985867 3963 raft_consensus.cc:729] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.985939 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf89d2bae6d8488c8c774557f4503365" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:31.986161 3773 raft_consensus.cc:2454] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:31.986120 3963 consensus_queue.cc:260] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:31.986558 3812 leader_election.cc:302] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:31.986794 3967 raft_consensus.cc:2781] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:31.986871 3967 raft_consensus.cc:482] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:31.986979 3967 raft_consensus.cc:3037] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:31.987213 3963 ts_tablet_manager.cc:1398] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:31.987462 3963 tablet_bootstrap.cc:492] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:31.987509 3964 tablet_bootstrap.cc:492] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:31.987634 3964 ts_tablet_manager.cc:1367] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20240503 02:13:31.988368 3964 raft_consensus.cc:348] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:31.988526 3964 raft_consensus.cc:374] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:31.988576 3964 raft_consensus.cc:729] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:31.988763 3964 consensus_queue.cc:260] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:31.988915 3963 tablet_bootstrap.cc:654] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.989969 3967 raft_consensus.cc:504] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.991024 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf89d2bae6d8488c8c774557f4503365" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:31.991235 3773 raft_consensus.cc:3037] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:31.991520 3964 ts_tablet_manager.cc:1398] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:31.994390 3964 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:31.994496 3969 raft_consensus.cc:482] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:31.994623 3969 raft_consensus.cc:504] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:31.995043 3773 raft_consensus.cc:2454] T cf89d2bae6d8488c8c774557f4503365 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:31.995211 3969 leader_election.cc:288] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:31.995404 3812 leader_election.cc:302] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:31.995636 3984 raft_consensus.cc:2781] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:31.995796 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ee6a0a5f584efe81cf64179fd6c4f0" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:31.995962 3848 raft_consensus.cc:2454] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:31.996212 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ee6a0a5f584efe81cf64179fd6c4f0" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:31.996308 3885 leader_election.cc:302] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:31.996397 3772 raft_consensus.cc:2454] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:31.996546 3969 raft_consensus.cc:2781] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:31.996632 3969 raft_consensus.cc:482] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:31.996733 3969 raft_consensus.cc:3037] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:31.996390 3964 tablet_bootstrap.cc:654] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:31.998073 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf89d2bae6d8488c8c774557f4503365" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:31.998541 3967 leader_election.cc:288] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
W20240503 02:13:31.998548 3814 leader_election.cc:341] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:31.999315 3962 tablet_bootstrap.cc:492] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:31.999466 3962 ts_tablet_manager.cc:1367] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.006s
I20240503 02:13:32.000047 3969 raft_consensus.cc:504] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.000411 3962 raft_consensus.cc:348] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:32.000650 3962 raft_consensus.cc:374] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.000720 3962 raft_consensus.cc:729] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.001096 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ee6a0a5f584efe81cf64179fd6c4f0" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.001026 3962 consensus_queue.cc:260] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:32.001353 3773 raft_consensus.cc:3037] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.001484 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ee6a0a5f584efe81cf64179fd6c4f0" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.001642 3848 raft_consensus.cc:3037] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.001869 3962 ts_tablet_manager.cc:1398] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.002198 3962 tablet_bootstrap.cc:492] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.004384 3848 raft_consensus.cc:2454] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.004824 3885 leader_election.cc:302] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.005085 3968 raft_consensus.cc:2781] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.005110 3969 leader_election.cc:288] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.005185 3962 tablet_bootstrap.cc:654] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.007836 3963 tablet_bootstrap.cc:492] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.007975 3963 ts_tablet_manager.cc:1367] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.003s
I20240503 02:13:32.008833 3963 raft_consensus.cc:348] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.009130 3963 raft_consensus.cc:374] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.009248 3963 raft_consensus.cc:729] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.009442 3975 raft_consensus.cc:482] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.009646 3975 raft_consensus.cc:504] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.009498 3963 consensus_queue.cc:260] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.010427 3975 leader_election.cc:288] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.010425 3963 ts_tablet_manager.cc:1398] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:32.010836 3963 tablet_bootstrap.cc:492] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.010965 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c6e46e6b9f47b199a76a6dc02d5a94" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.011121 3848 raft_consensus.cc:2454] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.011348 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c6e46e6b9f47b199a76a6dc02d5a94" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.011507 3922 raft_consensus.cc:2454] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.011899 3739 leader_election.cc:302] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.012424 3963 tablet_bootstrap.cc:654] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.012763 3975 raft_consensus.cc:2781] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.012838 3975 raft_consensus.cc:482] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.012897 3975 raft_consensus.cc:3037] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.015413 3963 tablet_bootstrap.cc:492] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.015524 3963 ts_tablet_manager.cc:1367] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:13:32.015620 3962 tablet_bootstrap.cc:492] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.015796 3962 ts_tablet_manager.cc:1367] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20240503 02:13:32.016211 3963 raft_consensus.cc:348] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.016366 3963 raft_consensus.cc:374] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.016407 3963 raft_consensus.cc:729] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.016557 3963 consensus_queue.cc:260] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.016594 3962 raft_consensus.cc:348] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.016813 3962 raft_consensus.cc:374] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.016868 3962 raft_consensus.cc:729] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.016921 3967 raft_consensus.cc:482] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.017015 3967 raft_consensus.cc:504] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.017108 3962 consensus_queue.cc:260] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.017517 3967 leader_election.cc:288] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.017758 3962 ts_tablet_manager.cc:1398] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.018043 3962 tablet_bootstrap.cc:492] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.018162 3963 ts_tablet_manager.cc:1398] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:32.018281 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.018414 3963 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
W20240503 02:13:32.018743 3812 leader_election.cc:341] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.019181 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:32.019618 3814 leader_election.cc:341] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.019713 3814 leader_election.cc:302] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.020028 3967 raft_consensus.cc:2726] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.020289 3963 tablet_bootstrap.cc:654] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.020447 3962 tablet_bootstrap.cc:654] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.021081 3975 raft_consensus.cc:504] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.022189 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c6e46e6b9f47b199a76a6dc02d5a94" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.022379 3922 raft_consensus.cc:3037] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.027027 3962 tablet_bootstrap.cc:492] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.027168 3962 ts_tablet_manager.cc:1367] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20240503 02:13:32.027833 3922 raft_consensus.cc:2454] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.027906 3962 raft_consensus.cc:348] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.029363 3962 raft_consensus.cc:374] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.029454 3962 raft_consensus.cc:729] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.029490 3963 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.028124 3975 leader_election.cc:288] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.029731 3963 ts_tablet_manager.cc:1367] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20240503 02:13:32.029832 3975 raft_consensus.cc:482] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.029927 3739 leader_election.cc:302] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.029932 3975 raft_consensus.cc:504] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:32.030722 3963 raft_consensus.cc:348] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.029744 3962 consensus_queue.cc:260] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.031015 3963 raft_consensus.cc:374] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.031088 3963 raft_consensus.cc:729] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.031267 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f36b48aefc45f88e8ad43c2a473cbe" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.031338 3963 consensus_queue.cc:260] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.031463 3962 ts_tablet_manager.cc:1398] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:32.031762 3975 leader_election.cc:288] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.031894 3975 raft_consensus.cc:2781] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.031993 3975 raft_consensus.cc:686] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:32.032085 3963 ts_tablet_manager.cc:1398] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.032199 3922 raft_consensus.cc:2454] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.032400 3963 tablet_bootstrap.cc:492] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.031994 3962 tablet_bootstrap.cc:492] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.032806 3739 leader_election.cc:302] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.034471 3967 raft_consensus.cc:482] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.034673 3963 tablet_bootstrap.cc:654] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.034629 3967 raft_consensus.cc:504] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.035344 3967 leader_election.cc:288] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.034452 3962 tablet_bootstrap.cc:654] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.036047 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:32.036584 3814 leader_election.cc:341] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.036741 3969 raft_consensus.cc:482] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.036865 3969 raft_consensus.cc:504] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.037600 3969 leader_election.cc:288] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.038156 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
W20240503 02:13:32.038652 3885 leader_election.cc:341] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.038632 4016 raft_consensus.cc:2781] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.039109 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.039199 4016 raft_consensus.cc:482] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.039269 4016 raft_consensus.cc:3037] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:32.039649 3886 leader_election.cc:341] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.039742 3886 leader_election.cc:302] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.040237 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.040437 3772 raft_consensus.cc:2454] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.040699 3969 raft_consensus.cc:2726] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.041272 3812 leader_election.cc:302] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.041733 3967 raft_consensus.cc:2781] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.041821 3967 raft_consensus.cc:482] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.041914 3967 raft_consensus.cc:3037] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.032249 3975 consensus_queue.cc:237] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.044682 3773 raft_consensus.cc:2454] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.045037 3967 raft_consensus.cc:504] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.045948 3969 raft_consensus.cc:482] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.046093 3969 raft_consensus.cc:504] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.046737 3969 leader_election.cc:288] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.047307 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f349445bf4c4fb9a6a38825241ee2ca" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.047468 3848 raft_consensus.cc:2454] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.047626 3967 leader_election.cc:288] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.047847 3885 leader_election.cc:302] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.048355 3969 raft_consensus.cc:2781] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.048420 3969 raft_consensus.cc:482] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.048482 3969 raft_consensus.cc:3037] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.051092 3969 raft_consensus.cc:504] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.051964 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f349445bf4c4fb9a6a38825241ee2ca" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.052151 3848 raft_consensus.cc:3037] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.051069 4016 raft_consensus.cc:504] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:32.054899 4016 leader_election.cc:288] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.055559 3848 raft_consensus.cc:2454] T 2f349445bf4c4fb9a6a38825241ee2ca P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.056028 3885 leader_election.cc:302] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.057312 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f349445bf4c4fb9a6a38825241ee2ca" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.057543 3772 raft_consensus.cc:2454] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.058275 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f36b48aefc45f88e8ad43c2a473cbe" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.058460 3921 raft_consensus.cc:3037] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.065431 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f349445bf4c4fb9a6a38825241ee2ca" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.065649 3772 raft_consensus.cc:3037] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.069353 3772 raft_consensus.cc:2454] T 2f349445bf4c4fb9a6a38825241ee2ca P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.070540 3921 raft_consensus.cc:2454] T b9f36b48aefc45f88e8ad43c2a473cbe P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.071190 3739 leader_election.cc:302] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.071483 4016 raft_consensus.cc:2781] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.071529 3969 leader_election.cc:288] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.071602 4016 raft_consensus.cc:686] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:32.071898 3969 raft_consensus.cc:2781] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.072044 3969 raft_consensus.cc:686] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.072005 4016 consensus_queue.cc:237] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } }
I20240503 02:13:32.072340 3969 consensus_queue.cc:237] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.073514 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
W20240503 02:13:32.074223 3814 leader_election.cc:341] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.075306 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.075565 3773 raft_consensus.cc:3037] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.079308 3773 raft_consensus.cc:2454] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.082479 3668 catalog_manager.cc:5484] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.095257 3812 leader_election.cc:302] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.095631 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c6e46e6b9f47b199a76a6dc02d5a94" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.095821 3848 raft_consensus.cc:3037] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.096051 3967 raft_consensus.cc:2781] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.096189 3967 raft_consensus.cc:686] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.096431 3967 consensus_queue.cc:237] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.098577 3848 raft_consensus.cc:2454] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.099656 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f36b48aefc45f88e8ad43c2a473cbe" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.099992 3848 raft_consensus.cc:2454] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.101348 3969 raft_consensus.cc:482] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.101222 3668 catalog_manager.cc:5484] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.101601 3969 raft_consensus.cc:504] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.102526 3967 raft_consensus.cc:482] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.102690 3967 raft_consensus.cc:504] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.103233 3967 leader_election.cc:288] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.103194 3847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.103447 3847 raft_consensus.cc:2454] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.103945 3885 leader_election.cc:302] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.106240 3969 leader_election.cc:288] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.106503 3969 raft_consensus.cc:2781] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.106681 3969 raft_consensus.cc:482] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.107131 3969 raft_consensus.cc:3037] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.106902 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:32.108690 3812 leader_election.cc:341] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.108752 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.109253 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f36b48aefc45f88e8ad43c2a473cbe" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.109614 3848 raft_consensus.cc:3037] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.113371 3848 raft_consensus.cc:2454] T b9f36b48aefc45f88e8ad43c2a473cbe P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.116449 3969 raft_consensus.cc:504] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.117049 3922 raft_consensus.cc:2379] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:32.117803 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.117915 3814 leader_election.cc:302] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
W20240503 02:13:32.118304 3886 leader_election.cc:341] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.118418 3967 raft_consensus.cc:3037] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.119056 3968 raft_consensus.cc:686] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.119419 3968 consensus_queue.cc:237] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.121117 3967 raft_consensus.cc:2726] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.122239 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.123267 3668 catalog_manager.cc:5484] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.124150 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
W20240503 02:13:32.124282 3886 leader_election.cc:341] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.126663 3969 leader_election.cc:288] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.130247 3668 catalog_manager.cc:5484] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.135699 3984 raft_consensus.cc:686] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.136161 3984 consensus_queue.cc:237] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.140556 3668 catalog_manager.cc:5484] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.147900 3668 catalog_manager.cc:5484] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.155082 3963 tablet_bootstrap.cc:492] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.155611 3848 raft_consensus.cc:2454] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.155638 3963 ts_tablet_manager.cc:1367] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.123s user 0.037s sys 0.083s
I20240503 02:13:32.156284 3885 leader_election.cc:302] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.157016 3969 raft_consensus.cc:2781] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.157198 3969 raft_consensus.cc:686] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.157276 3964 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.157402 3964 ts_tablet_manager.cc:1367] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.163s user 0.071s sys 0.075s
I20240503 02:13:32.157485 3969 consensus_queue.cc:237] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.157999 3964 raft_consensus.cc:348] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.158221 3964 raft_consensus.cc:374] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.158358 3964 raft_consensus.cc:729] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.158638 3964 consensus_queue.cc:260] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.159431 3964 ts_tablet_manager.cc:1398] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20240503 02:13:32.160023 3964 tablet_bootstrap.cc:492] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.157018 3963 raft_consensus.cc:348] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.161046 3963 raft_consensus.cc:374] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.161150 3963 raft_consensus.cc:729] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.161392 3963 consensus_queue.cc:260] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.161913 3963 ts_tablet_manager.cc:1398] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20240503 02:13:32.162057 3964 tablet_bootstrap.cc:654] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.162235 3963 tablet_bootstrap.cc:492] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.164139 3963 tablet_bootstrap.cc:654] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.164777 3962 tablet_bootstrap.cc:492] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.164914 3962 ts_tablet_manager.cc:1367] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.133s user 0.025s sys 0.062s
I20240503 02:13:32.165427 3669 catalog_manager.cc:5484] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.165661 3962 raft_consensus.cc:348] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.165809 3962 raft_consensus.cc:374] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.165884 3962 raft_consensus.cc:729] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.166081 3962 consensus_queue.cc:260] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.166551 3962 ts_tablet_manager.cc:1398] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:32.168767 3964 tablet_bootstrap.cc:492] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.168885 3962 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.168972 3964 ts_tablet_manager.cc:1367] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20240503 02:13:32.169947 3964 raft_consensus.cc:348] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.170259 3964 raft_consensus.cc:374] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.170358 3964 raft_consensus.cc:729] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.170545 3962 tablet_bootstrap.cc:654] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.170650 3964 consensus_queue.cc:260] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.171366 3964 ts_tablet_manager.cc:1398] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:32.171703 3964 tablet_bootstrap.cc:492] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.173743 3964 tablet_bootstrap.cc:654] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.175925 3963 tablet_bootstrap.cc:492] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.176060 3963 ts_tablet_manager.cc:1367] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20240503 02:13:32.176288 3964 tablet_bootstrap.cc:492] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.176446 3964 ts_tablet_manager.cc:1367] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:13:32.176749 3963 raft_consensus.cc:348] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.176918 3963 raft_consensus.cc:374] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.177023 3963 raft_consensus.cc:729] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.177085 3964 raft_consensus.cc:348] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.177243 3964 raft_consensus.cc:374] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.177371 3964 raft_consensus.cc:729] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.177313 3963 consensus_queue.cc:260] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.177605 3964 consensus_queue.cc:260] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.178217 3964 ts_tablet_manager.cc:1398] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.178489 3964 tablet_bootstrap.cc:492] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.178934 3963 ts_tablet_manager.cc:1398] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:32.179292 3963 tablet_bootstrap.cc:492] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.180016 3964 tablet_bootstrap.cc:654] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.180972 3963 tablet_bootstrap.cc:654] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.182034 3964 tablet_bootstrap.cc:492] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.182188 3964 ts_tablet_manager.cc:1367] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20240503 02:13:32.182940 3963 tablet_bootstrap.cc:492] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.182876 3964 raft_consensus.cc:348] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.183085 3964 raft_consensus.cc:374] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.183192 3964 raft_consensus.cc:729] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.183215 3963 ts_tablet_manager.cc:1367] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:13:32.183462 3964 consensus_queue.cc:260] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.183794 3962 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.183959 3964 ts_tablet_manager.cc:1398] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:32.183964 3962 ts_tablet_manager.cc:1367] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20240503 02:13:32.184338 3964 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.184196 3963 raft_consensus.cc:348] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.184554 3963 raft_consensus.cc:374] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.184688 3963 raft_consensus.cc:729] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.184801 3962 raft_consensus.cc:348] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.185106 3962 raft_consensus.cc:374] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.185236 3962 raft_consensus.cc:729] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.185067 3963 consensus_queue.cc:260] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.185488 3962 consensus_queue.cc:260] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.185976 3962 ts_tablet_manager.cc:1398] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.186026 3964 tablet_bootstrap.cc:654] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.186376 3962 tablet_bootstrap.cc:492] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.186630 3963 ts_tablet_manager.cc:1398] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:32.186996 3963 tablet_bootstrap.cc:492] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.187933 3962 tablet_bootstrap.cc:654] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.189229 3963 tablet_bootstrap.cc:654] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.195197 3962 tablet_bootstrap.cc:492] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.195356 3962 ts_tablet_manager.cc:1367] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20240503 02:13:32.196081 3962 raft_consensus.cc:348] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.196370 3962 raft_consensus.cc:374] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.196450 3962 raft_consensus.cc:729] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.196640 3962 consensus_queue.cc:260] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.197106 3962 ts_tablet_manager.cc:1398] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.197412 3962 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.197712 3963 tablet_bootstrap.cc:492] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.197930 3963 ts_tablet_manager.cc:1367] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20240503 02:13:32.198881 3962 tablet_bootstrap.cc:654] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.200932 3964 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.201043 3964 ts_tablet_manager.cc:1367] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.012s
I20240503 02:13:32.201630 3964 raft_consensus.cc:348] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.201781 3964 raft_consensus.cc:374] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.201828 3964 raft_consensus.cc:729] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.202034 3964 consensus_queue.cc:260] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.203310 3964 ts_tablet_manager.cc:1398] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.203552 3964 tablet_bootstrap.cc:492] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.204895 3964 tablet_bootstrap.cc:654] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.207340 3962 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.207432 3962 ts_tablet_manager.cc:1367] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20240503 02:13:32.207907 3962 raft_consensus.cc:348] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.208036 3962 raft_consensus.cc:374] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.208071 3962 raft_consensus.cc:729] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.208226 3962 consensus_queue.cc:260] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.208658 3962 ts_tablet_manager.cc:1398] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240503 02:13:32.208971 3962 tablet_bootstrap.cc:492] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.209609 3984 raft_consensus.cc:482] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.209764 3984 raft_consensus.cc:504] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.210443 3962 tablet_bootstrap.cc:654] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.210433 3984 leader_election.cc:288] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.210920 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141b1d61e275407a9827ce8ba0bd1913" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.211139 3922 raft_consensus.cc:2454] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.211565 3814 leader_election.cc:302] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.211822 3984 raft_consensus.cc:2781] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.211894 3984 raft_consensus.cc:482] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.211964 3984 raft_consensus.cc:3037] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.212181 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141b1d61e275407a9827ce8ba0bd1913" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.212344 3773 raft_consensus.cc:2454] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.214596 3984 raft_consensus.cc:504] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.215158 3984 leader_election.cc:288] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.215659 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141b1d61e275407a9827ce8ba0bd1913" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.215798 3922 raft_consensus.cc:3037] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.217339 3922 raft_consensus.cc:2454] T 141b1d61e275407a9827ce8ba0bd1913 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.217896 3814 leader_election.cc:302] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.217914 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "141b1d61e275407a9827ce8ba0bd1913" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.218394 3964 tablet_bootstrap.cc:492] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.218467 3773 raft_consensus.cc:3037] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.218508 3964 ts_tablet_manager.cc:1367] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20240503 02:13:32.219192 3964 raft_consensus.cc:348] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.219374 3964 raft_consensus.cc:374] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.219434 3964 raft_consensus.cc:729] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.219326 3963 raft_consensus.cc:348] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.219676 3963 raft_consensus.cc:374] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.219656 3964 consensus_queue.cc:260] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.219766 3963 raft_consensus.cc:729] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.220058 3963 consensus_queue.cc:260] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.221065 3964 ts_tablet_manager.cc:1398] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.221134 3963 ts_tablet_manager.cc:1398] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:32.221339 3964 tablet_bootstrap.cc:492] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.221454 3963 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.221980 3984 raft_consensus.cc:2781] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.222231 3984 raft_consensus.cc:686] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.222702 3963 tablet_bootstrap.cc:654] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.222725 3964 tablet_bootstrap.cc:654] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.223551 3773 raft_consensus.cc:2454] T 141b1d61e275407a9827ce8ba0bd1913 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.222548 3984 consensus_queue.cc:237] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.224118 3962 tablet_bootstrap.cc:492] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.224236 3962 ts_tablet_manager.cc:1367] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.004s
I20240503 02:13:32.224932 3962 raft_consensus.cc:348] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.225095 3962 raft_consensus.cc:374] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.225159 3962 raft_consensus.cc:729] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.225330 3962 consensus_queue.cc:260] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.225883 3962 ts_tablet_manager.cc:1398] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:32.226145 3962 tablet_bootstrap.cc:492] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.227281 3962 tablet_bootstrap.cc:654] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.231009 3962 tablet_bootstrap.cc:492] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.231128 3962 ts_tablet_manager.cc:1367] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240503 02:13:32.231644 3962 raft_consensus.cc:348] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.231806 3962 raft_consensus.cc:374] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.231869 3962 raft_consensus.cc:729] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.232074 3962 consensus_queue.cc:260] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.232647 3962 ts_tablet_manager.cc:1398] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240503 02:13:32.232899 3962 tablet_bootstrap.cc:492] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.234606 3962 tablet_bootstrap.cc:654] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.238281 3962 tablet_bootstrap.cc:492] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.238402 3962 ts_tablet_manager.cc:1367] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240503 02:13:32.238430 3984 raft_consensus.cc:482] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.238555 3984 raft_consensus.cc:504] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.239444 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.239143 3984 leader_election.cc:288] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.239132 3671 catalog_manager.cc:5484] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.238950 3962 raft_consensus.cc:348] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.239887 3962 raft_consensus.cc:374] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 02:13:32.239887 3814 leader_election.cc:341] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.239411 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.240046 3962 raft_consensus.cc:729] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.240166 3773 raft_consensus.cc:2454] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.240435 3962 consensus_queue.cc:260] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.241003 3962 ts_tablet_manager.cc:1398] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:13:32.241228 3962 tablet_bootstrap.cc:492] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.241335 3812 leader_election.cc:302] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.242300 3984 raft_consensus.cc:2781] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.242388 3984 raft_consensus.cc:482] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.242448 3962 tablet_bootstrap.cc:654] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.242480 3984 raft_consensus.cc:3037] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.244122 3962 tablet_bootstrap.cc:492] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.244232 3962 ts_tablet_manager.cc:1367] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:32.244786 3984 raft_consensus.cc:504] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.244848 3962 raft_consensus.cc:348] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.244973 3962 raft_consensus.cc:374] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.245036 3962 raft_consensus.cc:729] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.245230 3962 consensus_queue.cc:260] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.245826 3962 ts_tablet_manager.cc:1398] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240503 02:13:32.245770 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.245906 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.246101 3773 raft_consensus.cc:3037] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.246276 3984 leader_election.cc:288] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
W20240503 02:13:32.246248 3814 leader_election.cc:341] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.247189 3963 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.247287 3963 ts_tablet_manager.cc:1367] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.013s
I20240503 02:13:32.247462 3962 tablet_bootstrap.cc:492] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.247934 3963 raft_consensus.cc:348] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.248093 3963 raft_consensus.cc:374] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.248152 3963 raft_consensus.cc:729] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.248348 3963 consensus_queue.cc:260] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.248657 3773 raft_consensus.cc:2454] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.248853 3962 tablet_bootstrap.cc:654] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.249341 3964 tablet_bootstrap.cc:492] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.249468 3964 ts_tablet_manager.cc:1367] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.014s
I20240503 02:13:32.250044 3964 raft_consensus.cc:348] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.250222 3964 raft_consensus.cc:374] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.250274 3964 raft_consensus.cc:729] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.250461 3964 consensus_queue.cc:260] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.250927 3964 ts_tablet_manager.cc:1398] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240503 02:13:32.251124 3963 ts_tablet_manager.cc:1398] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:32.251170 3964 tablet_bootstrap.cc:492] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.252358 3964 tablet_bootstrap.cc:654] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.253886 3812 leader_election.cc:302] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.254350 4008 raft_consensus.cc:482] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.254487 4008 raft_consensus.cc:504] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.254731 3963 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.255198 4008 leader_election.cc:288] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.255369 4008 raft_consensus.cc:2781] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.255498 4008 raft_consensus.cc:686] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.255761 4008 consensus_queue.cc:237] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.256281 3963 tablet_bootstrap.cc:654] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.257613 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.257834 3922 raft_consensus.cc:2454] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.258298 3814 leader_election.cc:302] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.258625 3967 raft_consensus.cc:2781] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.258741 3967 raft_consensus.cc:482] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.258849 3967 raft_consensus.cc:3037] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.265551 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:32.266185 3812 leader_election.cc:341] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.269129 3967 raft_consensus.cc:504] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.270368 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.270601 3922 raft_consensus.cc:3037] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.273762 3967 leader_election.cc:288] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.274386 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:32.274924 3812 leader_election.cc:341] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.275713 3922 raft_consensus.cc:2454] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.276141 3814 leader_election.cc:302] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.276684 3984 raft_consensus.cc:2781] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.276823 3984 raft_consensus.cc:686] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.277170 3984 consensus_queue.cc:237] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.288332 3671 catalog_manager.cc:5484] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.288930 4013 raft_consensus.cc:482] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.289098 4013 raft_consensus.cc:504] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.289945 4013 leader_election.cc:288] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.291862 3962 tablet_bootstrap.cc:492] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.292001 3962 ts_tablet_manager.cc:1367] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.045s user 0.024s sys 0.019s
I20240503 02:13:32.292160 3963 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.292279 3963 ts_tablet_manager.cc:1367] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.039s user 0.016s sys 0.020s
I20240503 02:13:32.292384 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.292485 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.292637 3848 raft_consensus.cc:2454] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.292711 3962 raft_consensus.cc:348] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.292853 3962 raft_consensus.cc:374] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.292904 3962 raft_consensus.cc:729] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.292904 3963 raft_consensus.cc:348] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:32.293020 3739 leader_election.cc:341] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.293097 3963 raft_consensus.cc:374] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.293145 3963 raft_consensus.cc:729] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.293177 3962 consensus_queue.cc:260] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.293354 3963 consensus_queue.cc:260] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.293792 3962 ts_tablet_manager.cc:1398] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.293843 3963 ts_tablet_manager.cc:1398] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240503 02:13:32.294025 3962 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.294458 3963 tablet_bootstrap.cc:492] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.294505 3671 catalog_manager.cc:5484] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.295308 3962 tablet_bootstrap.cc:654] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.295442 3963 tablet_bootstrap.cc:654] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.297137 3736 leader_election.cc:302] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.297310 3962 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.297403 3962 ts_tablet_manager.cc:1367] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240503 02:13:32.297550 4013 raft_consensus.cc:2781] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.297609 4013 raft_consensus.cc:482] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.297663 4013 raft_consensus.cc:3037] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.298002 3962 raft_consensus.cc:348] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.298192 3962 raft_consensus.cc:374] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.298270 3962 raft_consensus.cc:729] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.298471 3962 consensus_queue.cc:260] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.298923 3963 tablet_bootstrap.cc:492] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.298972 3962 ts_tablet_manager.cc:1398] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240503 02:13:32.299080 3963 ts_tablet_manager.cc:1367] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240503 02:13:32.299281 3962 tablet_bootstrap.cc:492] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.299574 3963 raft_consensus.cc:348] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.299748 3963 raft_consensus.cc:374] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.299851 3963 raft_consensus.cc:729] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.300055 3963 consensus_queue.cc:260] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.300228 4013 raft_consensus.cc:504] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.300524 3963 ts_tablet_manager.cc:1398] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:13:32.300596 3962 tablet_bootstrap.cc:654] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.300884 3963 tablet_bootstrap.cc:492] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.301234 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
W20240503 02:13:32.301822 3739 leader_election.cc:341] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.302184 3963 tablet_bootstrap.cc:654] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.302362 4008 raft_consensus.cc:482] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.302502 4008 raft_consensus.cc:504] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.303176 4008 leader_election.cc:288] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.303642 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.303861 3848 raft_consensus.cc:3037] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.305905 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e4e31c162b4f5aabed85d5183c7ad5" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:32.306464 3812 leader_election.cc:341] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.307128 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e4e31c162b4f5aabed85d5183c7ad5" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.309993 3963 tablet_bootstrap.cc:492] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
W20240503 02:13:32.310205 3814 leader_election.cc:341] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.310331 3814 leader_election.cc:302] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.310731 4008 raft_consensus.cc:2726] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.310950 3963 ts_tablet_manager.cc:1367] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20240503 02:13:32.311688 3963 raft_consensus.cc:348] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.311900 3963 raft_consensus.cc:374] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.311977 3963 raft_consensus.cc:729] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.312287 3963 consensus_queue.cc:260] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.312907 3963 ts_tablet_manager.cc:1398] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:13:32.313258 3963 tablet_bootstrap.cc:492] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.313318 4016 raft_consensus.cc:482] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.313465 4016 raft_consensus.cc:504] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.314051 4016 leader_election.cc:288] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.314631 3848 raft_consensus.cc:2454] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.314663 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb3ffa08c84ffbbd779d35a68ff7f6" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.314851 3922 raft_consensus.cc:2454] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.314857 4013 leader_election.cc:288] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.315040 3963 tablet_bootstrap.cc:654] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.315200 3964 tablet_bootstrap.cc:492] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.315363 3739 leader_election.cc:302] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.315493 3736 leader_election.cc:302] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.315651 4013 raft_consensus.cc:2781] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.315790 4013 raft_consensus.cc:482] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.315833 4016 raft_consensus.cc:2781] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.315860 3964 ts_tablet_manager.cc:1367] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.064s user 0.019s sys 0.017s
I20240503 02:13:32.315904 4013 raft_consensus.cc:3037] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.315994 4016 raft_consensus.cc:686] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:32.316248 4016 consensus_queue.cc:237] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.316565 3964 raft_consensus.cc:348] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.316756 3964 raft_consensus.cc:374] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.316833 3964 raft_consensus.cc:729] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.317040 3964 consensus_queue.cc:260] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.317581 3964 ts_tablet_manager.cc:1398] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:32.317977 3964 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.318434 4013 raft_consensus.cc:504] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.318922 4013 leader_election.cc:288] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.315647 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb3ffa08c84ffbbd779d35a68ff7f6" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.319198 3848 raft_consensus.cc:2454] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.319826 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb3ffa08c84ffbbd779d35a68ff7f6" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.320032 3922 raft_consensus.cc:3037] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.320169 3964 tablet_bootstrap.cc:654] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.322327 3964 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.322480 3964 ts_tablet_manager.cc:1367] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20240503 02:13:32.322929 3922 raft_consensus.cc:2454] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.323064 3964 raft_consensus.cc:348] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.323237 3963 tablet_bootstrap.cc:492] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.323264 3964 raft_consensus.cc:374] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.323377 3739 leader_election.cc:302] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.323381 3964 raft_consensus.cc:729] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.323487 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb3ffa08c84ffbbd779d35a68ff7f6" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.323678 3962 tablet_bootstrap.cc:492] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.323715 4013 raft_consensus.cc:2781] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.323683 3848 raft_consensus.cc:3037] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.323840 3962 ts_tablet_manager.cc:1367] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.004s
I20240503 02:13:32.323863 4013 raft_consensus.cc:686] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:32.323710 3964 consensus_queue.cc:260] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.323386 3963 ts_tablet_manager.cc:1367] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240503 02:13:32.324133 4013 consensus_queue.cc:237] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.324548 3962 raft_consensus.cc:348] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.324723 3962 raft_consensus.cc:374] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.324826 3962 raft_consensus.cc:729] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.324777 3963 raft_consensus.cc:348] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.324956 3963 raft_consensus.cc:374] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.325057 3963 raft_consensus.cc:729] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.325057 3962 consensus_queue.cc:260] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.325290 3963 consensus_queue.cc:260] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.325686 3962 ts_tablet_manager.cc:1398] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:32.325929 3963 ts_tablet_manager.cc:1398] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:32.326176 3963 tablet_bootstrap.cc:492] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.326449 3848 raft_consensus.cc:2454] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.327764 3963 tablet_bootstrap.cc:654] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.328187 3962 tablet_bootstrap.cc:492] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.331324 3962 tablet_bootstrap.cc:654] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.333039 3922 raft_consensus.cc:3037] T e1e8091f13274e0e9fd5eb58eedb6f9c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.335362 3669 catalog_manager.cc:5484] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.339746 3984 raft_consensus.cc:482] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.339988 3984 raft_consensus.cc:504] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.340958 3984 leader_election.cc:288] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.341200 3984 consensus_queue.cc:1035] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.341962 3962 tablet_bootstrap.cc:492] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.342151 3962 ts_tablet_manager.cc:1367] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20240503 02:13:32.342252 3963 tablet_bootstrap.cc:492] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.342191 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948ee5eebd9c405fa85066b3a57ebe02" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.342434 3963 ts_tablet_manager.cc:1367] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.003s
W20240503 02:13:32.342712 3812 leader_election.cc:341] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.342955 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948ee5eebd9c405fa85066b3a57ebe02" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.342955 3962 raft_consensus.cc:348] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.343230 3962 raft_consensus.cc:374] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.343142 3963 raft_consensus.cc:348] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.343269 3922 raft_consensus.cc:2454] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.343320 3962 raft_consensus.cc:729] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.343380 3963 raft_consensus.cc:374] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.343462 3963 raft_consensus.cc:729] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.343597 3962 consensus_queue.cc:260] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.343696 3963 consensus_queue.cc:260] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.344166 3962 ts_tablet_manager.cc:1398] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.344559 3962 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.345908 3963 ts_tablet_manager.cc:1398] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:32.345579 3814 leader_election.cc:302] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.346201 3963 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.346529 3984 raft_consensus.cc:2781] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.346684 3984 raft_consensus.cc:482] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.346800 3984 raft_consensus.cc:3037] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.347141 3962 tablet_bootstrap.cc:654] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.347760 3963 tablet_bootstrap.cc:654] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.350452 3984 raft_consensus.cc:504] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.351727 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948ee5eebd9c405fa85066b3a57ebe02" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:32.352344 3812 leader_election.cc:341] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.352735 3984 leader_election.cc:288] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.353713 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948ee5eebd9c405fa85066b3a57ebe02" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.353925 3921 raft_consensus.cc:3037] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.366271 4013 raft_consensus.cc:482] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.366483 4013 raft_consensus.cc:504] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.367342 4013 leader_election.cc:288] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.368161 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.368423 3922 raft_consensus.cc:2454] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.368948 3739 leader_election.cc:302] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.369410 4013 raft_consensus.cc:2781] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.369560 4013 raft_consensus.cc:482] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.369681 4013 raft_consensus.cc:3037] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.373239 4013 raft_consensus.cc:504] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.374428 4008 raft_consensus.cc:482] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.374608 4008 raft_consensus.cc:504] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.375380 4008 leader_election.cc:288] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.376298 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.378170 4016 raft_consensus.cc:482] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.378373 4016 raft_consensus.cc:504] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.379148 4016 leader_election.cc:288] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.379953 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.380121 3967 consensus_queue.cc:1035] T e1e8091f13274e0e9fd5eb58eedb6f9c P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.380808 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.381002 3848 raft_consensus.cc:2454] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:32.382431 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.382642 3922 raft_consensus.cc:2454] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.382988 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f3249a7b59413792a08471c2ae4328" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.383249 3848 raft_consensus.cc:2379] T 05f3249a7b59413792a08471c2ae4328 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:32.383575 3814 leader_election.cc:302] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.383723 3736 leader_election.cc:302] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.383831 3967 raft_consensus.cc:2781] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.383973 3967 raft_consensus.cc:482] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.384083 3967 raft_consensus.cc:3037] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.384213 4016 raft_consensus.cc:3037] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.387921 4016 raft_consensus.cc:2726] T 05f3249a7b59413792a08471c2ae4328 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.389370 3967 raft_consensus.cc:504] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.392414 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.393424 3967 leader_election.cc:288] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.357510 3921 raft_consensus.cc:2454] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.393746 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.393944 3922 raft_consensus.cc:3037] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.394160 3773 raft_consensus.cc:2379] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in current term 1: Already voted for candidate 6e0d1128f0404d609df72fd607c20650 in this term.
I20240503 02:13:32.395365 4013 leader_election.cc:288] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.395635 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.396569 3922 raft_consensus.cc:2454] T 02fb95baa11649fdbf58393d0de53fff P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.397080 3814 leader_election.cc:302] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.397080 3671 catalog_manager.cc:5484] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.397501 3814 leader_election.cc:302] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.397707 3984 raft_consensus.cc:2781] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.400336 3984 raft_consensus.cc:686] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.400507 3967 raft_consensus.cc:2781] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.400626 3984 consensus_queue.cc:237] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.397821 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02fb95baa11649fdbf58393d0de53fff" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.410166 3967 raft_consensus.cc:686] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.410512 3967 consensus_queue.cc:237] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.411408 3963 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.411600 3963 ts_tablet_manager.cc:1367] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.065s user 0.029s sys 0.037s
I20240503 02:13:32.411944 3964 ts_tablet_manager.cc:1398] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.089s user 0.025s sys 0.025s
I20240503 02:13:32.412242 3962 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.412279 3964 tablet_bootstrap.cc:492] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.412456 3962 ts_tablet_manager.cc:1367] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.068s user 0.028s sys 0.027s
I20240503 02:13:32.412456 3963 raft_consensus.cc:348] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.412747 3963 raft_consensus.cc:374] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.412850 3963 raft_consensus.cc:729] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.413112 3963 consensus_queue.cc:260] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.413245 3962 raft_consensus.cc:348] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.413424 3962 raft_consensus.cc:374] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.413566 3962 raft_consensus.cc:729] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.413844 3964 tablet_bootstrap.cc:654] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.413774 3962 consensus_queue.cc:260] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.414392 3962 ts_tablet_manager.cc:1398] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:32.414695 3962 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.415505 3963 ts_tablet_manager.cc:1398] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:32.415870 3963 tablet_bootstrap.cc:492] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.416453 3671 catalog_manager.cc:5484] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.417178 3671 catalog_manager.cc:5484] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.417987 3963 tablet_bootstrap.cc:654] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.426101 3736 leader_election.cc:302] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.426425 4056 raft_consensus.cc:2726] T 02fb95baa11649fdbf58393d0de53fff P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.426460 3963 tablet_bootstrap.cc:492] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.426648 3963 ts_tablet_manager.cc:1367] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20240503 02:13:32.426913 3962 tablet_bootstrap.cc:654] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.427407 3963 raft_consensus.cc:348] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.427681 3963 raft_consensus.cc:374] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.427785 3963 raft_consensus.cc:729] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.428123 3963 consensus_queue.cc:260] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.429220 3963 ts_tablet_manager.cc:1398] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.429633 3963 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.431723 3963 tablet_bootstrap.cc:654] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.433306 3969 raft_consensus.cc:482] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.433476 3969 raft_consensus.cc:504] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.434068 3969 leader_election.cc:288] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.434384 3969 raft_consensus.cc:482] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.434590 3969 raft_consensus.cc:504] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.434896 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ea517f608bc44a3a34d6fa0fa37e147" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.435101 3773 raft_consensus.cc:2454] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.435359 3969 leader_election.cc:288] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.435684 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ea517f608bc44a3a34d6fa0fa37e147" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.435954 3848 raft_consensus.cc:2454] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.436053 3886 leader_election.cc:302] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.436401 3969 raft_consensus.cc:2781] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.436580 3969 raft_consensus.cc:482] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.436712 3969 raft_consensus.cc:3037] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.437378 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.441394 3773 raft_consensus.cc:2379] T ba261cc9d5784f15aaeeb87e8a7cae51 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4001ccb96b064ad2a2ca9c61096516fe in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:32.442164 3969 raft_consensus.cc:504] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.443588 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba261cc9d5784f15aaeeb87e8a7cae51" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.444852 3885 leader_election.cc:302] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.445170 3968 raft_consensus.cc:3037] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.445729 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ea517f608bc44a3a34d6fa0fa37e147" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.445899 3848 raft_consensus.cc:3037] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.447964 3968 raft_consensus.cc:2726] T ba261cc9d5784f15aaeeb87e8a7cae51 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.448501 3848 raft_consensus.cc:2454] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.448895 3885 leader_election.cc:302] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.448937 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ea517f608bc44a3a34d6fa0fa37e147" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.449208 3968 raft_consensus.cc:2781] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.449420 3968 raft_consensus.cc:686] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.449720 3968 consensus_queue.cc:237] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.450369 3962 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.450556 3962 ts_tablet_manager.cc:1367] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.013s
I20240503 02:13:32.450855 3963 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.449216 3772 raft_consensus.cc:3037] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.451179 3963 ts_tablet_manager.cc:1367] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.007s
I20240503 02:13:32.451311 3969 leader_election.cc:288] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.451370 3962 raft_consensus.cc:348] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.451594 3962 raft_consensus.cc:374] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.451683 3962 raft_consensus.cc:729] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.451942 3963 raft_consensus.cc:348] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.451920 3962 consensus_queue.cc:260] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.452091 3963 raft_consensus.cc:374] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.452221 3963 raft_consensus.cc:729] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.452463 3962 ts_tablet_manager.cc:1398] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:32.452440 3963 consensus_queue.cc:260] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.452752 3962 tablet_bootstrap.cc:492] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.453011 3963 ts_tablet_manager.cc:1398] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.453253 3963 tablet_bootstrap.cc:492] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.453980 3772 raft_consensus.cc:2454] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.455264 3962 tablet_bootstrap.cc:654] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.456595 3964 tablet_bootstrap.cc:492] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.456763 3964 ts_tablet_manager.cc:1367] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.045s user 0.008s sys 0.004s
I20240503 02:13:32.456416 3668 catalog_manager.cc:5484] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.457495 3964 raft_consensus.cc:348] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.457739 3964 raft_consensus.cc:374] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.457836 3964 raft_consensus.cc:729] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.458092 3964 consensus_queue.cc:260] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.458626 3964 ts_tablet_manager.cc:1398] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.459049 3963 tablet_bootstrap.cc:654] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.459724 3962 tablet_bootstrap.cc:492] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.459780 3968 raft_consensus.cc:482] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.459971 3962 ts_tablet_manager.cc:1367] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20240503 02:13:32.459933 3968 raft_consensus.cc:504] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.460685 3968 leader_election.cc:288] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.461128 3962 raft_consensus.cc:348] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.461454 3962 raft_consensus.cc:374] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.461649 3962 raft_consensus.cc:729] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.462023 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b76a1a9dea45aebd7fea02fbc9cb42" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.461979 3962 consensus_queue.cc:260] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.462234 3772 raft_consensus.cc:2454] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.462563 3886 leader_election.cc:302] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.462687 3962 ts_tablet_manager.cc:1398] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.462817 3968 raft_consensus.cc:2781] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.462921 3968 raft_consensus.cc:482] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.463016 3962 tablet_bootstrap.cc:492] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.463050 3968 raft_consensus.cc:3037] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.465245 3962 tablet_bootstrap.cc:654] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.465860 3968 raft_consensus.cc:504] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.466811 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b76a1a9dea45aebd7fea02fbc9cb42" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.461447 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b76a1a9dea45aebd7fea02fbc9cb42" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.467002 3772 raft_consensus.cc:3037] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.467142 3848 raft_consensus.cc:2454] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.469281 3963 tablet_bootstrap.cc:492] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.469478 3963 ts_tablet_manager.cc:1367] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.003s
I20240503 02:13:32.469919 3772 raft_consensus.cc:2454] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.470268 3886 leader_election.cc:302] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.470458 3963 raft_consensus.cc:348] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.470628 3969 raft_consensus.cc:2781] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.470813 3969 raft_consensus.cc:686] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.470945 3968 leader_election.cc:288] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.460074 3964 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.471156 3969 consensus_queue.cc:237] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.470784 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23b76a1a9dea45aebd7fea02fbc9cb42" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.472386 3848 raft_consensus.cc:3037] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.472514 3964 tablet_bootstrap.cc:654] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.474264 3964 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.474377 3964 ts_tablet_manager.cc:1367] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.001s
I20240503 02:13:32.474922 3964 raft_consensus.cc:348] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.475065 3964 raft_consensus.cc:374] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.475129 3848 raft_consensus.cc:2454] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.475142 3964 raft_consensus.cc:729] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.475492 3964 consensus_queue.cc:260] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.475993 3962 tablet_bootstrap.cc:492] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.476219 3962 ts_tablet_manager.cc:1367] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.003s
I20240503 02:13:32.476181 3964 ts_tablet_manager.cc:1398] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.476563 3964 tablet_bootstrap.cc:492] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.477007 3962 raft_consensus.cc:348] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.477277 3962 raft_consensus.cc:374] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.477371 3962 raft_consensus.cc:729] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.477408 3668 catalog_manager.cc:5484] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.477627 3962 consensus_queue.cc:260] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.477900 3963 raft_consensus.cc:374] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.477988 3963 raft_consensus.cc:729] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.478226 3963 consensus_queue.cc:260] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.478406 3964 tablet_bootstrap.cc:654] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.478875 3963 ts_tablet_manager.cc:1398] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20240503 02:13:32.479164 3963 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.480306 3962 ts_tablet_manager.cc:1398] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:32.480659 3962 tablet_bootstrap.cc:492] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.480692 3963 tablet_bootstrap.cc:654] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.482435 3962 tablet_bootstrap.cc:654] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.482906 3967 raft_consensus.cc:482] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.483105 3967 raft_consensus.cc:504] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.483901 3967 leader_election.cc:288] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.484586 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab19c51fbe2e4417b2fb32f20d51d5bc" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.484833 3922 raft_consensus.cc:2454] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.485635 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab19c51fbe2e4417b2fb32f20d51d5bc" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.486330 3814 leader_election.cc:302] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
W20240503 02:13:32.486739 3812 leader_election.cc:341] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.487783 3772 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.487890 3962 tablet_bootstrap.cc:492] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.488065 3962 ts_tablet_manager.cc:1367] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20240503 02:13:32.489140 3962 raft_consensus.cc:348] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.489500 3962 raft_consensus.cc:374] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.491729 3962 raft_consensus.cc:729] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.492039 3962 consensus_queue.cc:260] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.492713 3962 ts_tablet_manager.cc:1398] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240503 02:13:32.493067 3962 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.493248 3969 consensus_queue.cc:1035] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.495164 3962 tablet_bootstrap.cc:654] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.495642 3967 raft_consensus.cc:2781] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.495556 4056 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.495744 3967 raft_consensus.cc:482] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.495847 3967 raft_consensus.cc:3037] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.497175 3963 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.497278 3963 ts_tablet_manager.cc:1367] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.012s
I20240503 02:13:32.498026 3963 raft_consensus.cc:348] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.498201 3963 raft_consensus.cc:374] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.498299 3963 raft_consensus.cc:729] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.498334 3962 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.498471 3962 ts_tablet_manager.cc:1367] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240503 02:13:32.498546 3963 consensus_queue.cc:260] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.499148 3963 ts_tablet_manager.cc:1398] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.499110 3962 raft_consensus.cc:348] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.499079 3967 raft_consensus.cc:504] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.499305 3962 raft_consensus.cc:374] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.499378 3962 raft_consensus.cc:729] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.499410 3963 tablet_bootstrap.cc:492] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.500154 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab19c51fbe2e4417b2fb32f20d51d5bc" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.500348 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab19c51fbe2e4417b2fb32f20d51d5bc" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.500530 3921 raft_consensus.cc:3037] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:32.500624 3812 leader_election.cc:341] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.499639 3962 consensus_queue.cc:260] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.501034 3963 tablet_bootstrap.cc:654] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.501317 3962 ts_tablet_manager.cc:1398] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:32.501636 3962 tablet_bootstrap.cc:492] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.503204 3921 raft_consensus.cc:2454] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.503536 3962 tablet_bootstrap.cc:654] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.503605 4056 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.503734 4056 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.504210 3969 raft_consensus.cc:482] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.504454 4056 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.504382 3969 raft_consensus.cc:504] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.505002 3969 leader_election.cc:288] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.505162 3814 leader_election.cc:302] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.505487 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.505625 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.505795 3848 raft_consensus.cc:2454] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:32.505470 4066 raft_consensus.cc:2781] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.506103 4066 raft_consensus.cc:686] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.506431 4066 consensus_queue.cc:237] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.507030 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.507596 3967 leader_election.cc:288] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.507934 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffacc2d4cc534452ad50a87526626004" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.508251 3736 leader_election.cc:302] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters: 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.512068 3848 raft_consensus.cc:2379] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4001ccb96b064ad2a2ca9c61096516fe in current term 1: Already voted for candidate 6e0d1128f0404d609df72fd607c20650 in this term.
I20240503 02:13:32.512470 3885 leader_election.cc:302] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.513223 3669 catalog_manager.cc:5484] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.514084 3969 raft_consensus.cc:3037] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.509919 3963 tablet_bootstrap.cc:492] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.514497 3963 ts_tablet_manager.cc:1367] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20240503 02:13:32.514605 3962 tablet_bootstrap.cc:492] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.514767 3962 ts_tablet_manager.cc:1367] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20240503 02:13:32.515436 3962 raft_consensus.cc:348] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.515604 3962 raft_consensus.cc:374] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.515710 3962 raft_consensus.cc:729] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.515921 3962 consensus_queue.cc:260] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.516566 3962 ts_tablet_manager.cc:1398] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.516835 3962 tablet_bootstrap.cc:492] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.515264 3963 raft_consensus.cc:348] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.518107 4066 raft_consensus.cc:482] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.518193 3963 raft_consensus.cc:374] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.518307 3963 raft_consensus.cc:729] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.518251 3968 consensus_queue.cc:1035] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.518523 3963 consensus_queue.cc:260] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.518802 3962 tablet_bootstrap.cc:654] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.518272 4066 raft_consensus.cc:504] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.519130 3963 ts_tablet_manager.cc:1398] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:13:32.519434 4066 leader_election.cc:288] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.519439 3963 tablet_bootstrap.cc:492] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.520087 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4094a9a8c23046b6a2f27c451d8347f3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.520270 3921 raft_consensus.cc:2454] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.520707 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4094a9a8c23046b6a2f27c451d8347f3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.520941 3773 raft_consensus.cc:2454] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.521500 3814 leader_election.cc:302] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.521978 3963 tablet_bootstrap.cc:654] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.522099 4066 raft_consensus.cc:2781] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.522204 4066 raft_consensus.cc:482] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.522328 4066 raft_consensus.cc:3037] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.524129 4013 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.528476 3969 raft_consensus.cc:2726] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.525421 4066 raft_consensus.cc:504] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.531008 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4094a9a8c23046b6a2f27c451d8347f3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.531299 3773 raft_consensus.cc:3037] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.531890 3922 raft_consensus.cc:3037] T cf89d2bae6d8488c8c774557f4503365 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.535360 3773 raft_consensus.cc:2454] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.535916 3812 leader_election.cc:302] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:32.536072 3984 consensus_queue.cc:1035] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.536234 4008 raft_consensus.cc:2781] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.542831 3963 tablet_bootstrap.cc:492] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.543011 3963 ts_tablet_manager.cc:1367] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.006s
I20240503 02:13:32.543812 3963 raft_consensus.cc:348] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.543875 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4094a9a8c23046b6a2f27c451d8347f3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.544010 3963 raft_consensus.cc:374] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.544124 3922 raft_consensus.cc:3037] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.544181 3963 raft_consensus.cc:729] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.544591 3963 consensus_queue.cc:260] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.545668 3962 tablet_bootstrap.cc:492] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.545670 3963 ts_tablet_manager.cc:1398] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240503 02:13:32.545892 3962 ts_tablet_manager.cc:1367] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.004s
I20240503 02:13:32.546000 3963 tablet_bootstrap.cc:492] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.546554 3962 raft_consensus.cc:348] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.546754 3962 raft_consensus.cc:374] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.546845 3962 raft_consensus.cc:729] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.547125 3962 consensus_queue.cc:260] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.547832 3962 ts_tablet_manager.cc:1398] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.547804 3963 tablet_bootstrap.cc:654] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.548107 3962 tablet_bootstrap.cc:492] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.548296 4066 leader_election.cc:288] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.549749 3962 tablet_bootstrap.cc:654] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.549964 4056 raft_consensus.cc:2781] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240503 02:13:32.550894 4056 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4001ccb96b064ad2a2ca9c61096516fe)
I20240503 02:13:32.551055 4056 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:32.547816 3922 raft_consensus.cc:2454] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.554643 4056 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.554643 4010 consensus_queue.cc:1035] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.555596 4056 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 2 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.555619 3847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.556457 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.557027 3739 leader_election.cc:302] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.557603 4056 raft_consensus.cc:2726] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240503 02:13:32.561084 3969 raft_consensus.cc:482] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.561288 3969 raft_consensus.cc:504] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.562367 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb8007da5342d29290016341c5999b" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.562683 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb8007da5342d29290016341c5999b" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.562958 3846 raft_consensus.cc:2454] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
W20240503 02:13:32.563050 3886 leader_election.cc:341] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.563370 3885 leader_election.cc:302] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.563660 4010 raft_consensus.cc:2781] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.563771 4010 raft_consensus.cc:482] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.563879 4010 raft_consensus.cc:3037] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.564288 3969 leader_election.cc:288] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.566584 4008 raft_consensus.cc:686] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.566860 4008 consensus_queue.cc:237] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.570194 3669 catalog_manager.cc:5484] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.574136 3962 tablet_bootstrap.cc:492] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.574308 3962 ts_tablet_manager.cc:1367] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.002s
I20240503 02:13:32.574131 3964 tablet_bootstrap.cc:492] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.575287 3964 ts_tablet_manager.cc:1367] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.099s user 0.004s sys 0.030s
I20240503 02:13:32.575946 3964 raft_consensus.cc:348] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.576159 3964 raft_consensus.cc:374] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.576236 3964 raft_consensus.cc:729] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.576495 3964 consensus_queue.cc:260] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.576709 4010 raft_consensus.cc:504] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.577271 4010 leader_election.cc:288] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.577468 4010 raft_consensus.cc:482] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.577649 4010 raft_consensus.cc:504] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.578327 3964 ts_tablet_manager.cc:1398] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:32.578267 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb8007da5342d29290016341c5999b" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.578631 3964 tablet_bootstrap.cc:492] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.578660 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.578850 3773 raft_consensus.cc:2379] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4001ccb96b064ad2a2ca9c61096516fe in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:32.578817 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb8007da5342d29290016341c5999b" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.579027 3846 raft_consensus.cc:3037] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:32.579327 3886 leader_election.cc:341] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.581211 3963 tablet_bootstrap.cc:492] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.581540 3963 ts_tablet_manager.cc:1367] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.001s
I20240503 02:13:32.582772 4056 consensus_queue.cc:1035] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.583752 3967 consensus_queue.cc:1035] T cf89d2bae6d8488c8c774557f4503365 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.582775 3962 raft_consensus.cc:348] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.584163 3964 tablet_bootstrap.cc:654] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.584818 3846 raft_consensus.cc:2454] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.585345 4010 leader_election.cc:288] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.585287 3885 leader_election.cc:302] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.585827 3962 raft_consensus.cc:374] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.585994 3962 raft_consensus.cc:729] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.586261 3962 consensus_queue.cc:260] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.585726 4010 raft_consensus.cc:2781] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.588176 3967 raft_consensus.cc:482] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.588330 3967 raft_consensus.cc:504] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.589118 3967 leader_election.cc:288] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.589805 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9a7019777d47c48621b1b40e7ef540" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.589936 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9a7019777d47c48621b1b40e7ef540" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.590023 3922 raft_consensus.cc:2454] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.590401 3814 leader_election.cc:302] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.590754 3967 raft_consensus.cc:2781] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20240503 02:13:32.590828 3812 leader_election.cc:341] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.590926 3967 raft_consensus.cc:482] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.591058 3967 raft_consensus.cc:3037] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.597772 3847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.598737 3962 ts_tablet_manager.cc:1398] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.017s user 0.001s sys 0.002s
I20240503 02:13:32.592356 3963 raft_consensus.cc:348] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.614012 3963 raft_consensus.cc:374] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.614202 3963 raft_consensus.cc:729] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.614684 3963 consensus_queue.cc:260] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.615288 3962 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.618031 3962 tablet_bootstrap.cc:654] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.619560 4010 raft_consensus.cc:686] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.619853 4010 consensus_queue.cc:237] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.627035 4094 consensus_queue.cc:1035] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20240503 02:13:32.627604 4066 raft_consensus.cc:482] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.629791 4008 raft_consensus.cc:482] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.630355 3984 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.633440 4008 raft_consensus.cc:504] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.633764 3984 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.634429 3984 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.634781 4066 raft_consensus.cc:504] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.637148 4106 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.638696 4056 consensus_queue.cc:1035] T b9f36b48aefc45f88e8ad43c2a473cbe P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.639843 4106 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.641777 4008 leader_election.cc:288] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.642140 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.645057 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.649863 3969 consensus_queue.cc:1035] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.650488 3969 raft_consensus.cc:482] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.650519 3885 leader_election.cc:302] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.650641 3969 raft_consensus.cc:504] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.650368 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.651552 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.650236 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31d78763218f4ea3af2c4f4620e3cde4" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.651719 3969 leader_election.cc:288] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.657711 4094 raft_consensus.cc:482] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.658180 4066 leader_election.cc:288] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.658726 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31d78763218f4ea3af2c4f4620e3cde4" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:32.666976 3812 leader_election.cc:341] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.677103 4094 raft_consensus.cc:504] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.677580 3922 raft_consensus.cc:2454] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
W20240503 02:13:32.677729 3814 leader_election.cc:341] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.678097 3814 leader_election.cc:341] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.678195 3814 leader_election.cc:302] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.678584 4066 raft_consensus.cc:2726] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.679579 4108 raft_consensus.cc:482] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.679807 4108 raft_consensus.cc:504] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.680158 4105 raft_consensus.cc:482] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.680317 4105 raft_consensus.cc:504] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.680611 4102 raft_consensus.cc:3037] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.683611 4094 leader_election.cc:288] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.683754 4110 consensus_queue.cc:1035] T 2f349445bf4c4fb9a6a38825241ee2ca P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.684619 4094 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.670831 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a88f9cd3fa40c88cc361f53c906d4e" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.685904 3848 raft_consensus.cc:2454] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.686672 3885 leader_election.cc:302] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.687100 4109 raft_consensus.cc:2781] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.692229 4109 raft_consensus.cc:482] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.694150 4109 raft_consensus.cc:3037] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.691123 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a88f9cd3fa40c88cc361f53c906d4e" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.691587 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d1771c9b104e52a24a0411f05f0f94" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.695802 3922 raft_consensus.cc:2454] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.688278 4110 consensus_queue.cc:1035] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20240503 02:13:32.691133 3812 leader_election.cc:341] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.696779 3812 leader_election.cc:341] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.696907 3812 leader_election.cc:302] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.692090 3968 consensus_queue.cc:1035] T 05f3249a7b59413792a08471c2ae4328 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20240503 02:13:32.697939 4106 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.698020 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.698105 4066 raft_consensus.cc:2726] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.698168 3846 raft_consensus.cc:2454] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.692101 3963 ts_tablet_manager.cc:1398] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.086s user 0.003s sys 0.011s
I20240503 02:13:32.698608 3963 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.701910 3814 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.702446 3814 leader_election.cc:302] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.702703 3984 raft_consensus.cc:2781] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.702772 3984 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.702829 3984 raft_consensus.cc:3037] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.703559 4008 raft_consensus.cc:2781] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.703634 4008 raft_consensus.cc:482] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.703691 4008 raft_consensus.cc:3037] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.705081 3963 tablet_bootstrap.cc:654] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.705314 3984 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.706400 4066 consensus_queue.cc:1035] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.706768 4066 consensus_queue.cc:1035] T ba261cc9d5784f15aaeeb87e8a7cae51 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.701943 3885 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.706831 3984 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.707383 4110 raft_consensus.cc:2781] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.707451 4110 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.707495 4110 raft_consensus.cc:3037] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.707545 4008 raft_consensus.cc:504] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.708295 4008 leader_election.cc:288] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.709726 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:32.710325 3812 leader_election.cc:341] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.710639 4094 raft_consensus.cc:482] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.710800 4094 raft_consensus.cc:504] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.711539 4094 leader_election.cc:288] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.712448 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89de6eace7d643e2a78385a67c771991" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:32.712929 3812 leader_election.cc:341] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.717029 4109 raft_consensus.cc:504] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.720650 4102 raft_consensus.cc:2726] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.721496 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.722892 4109 leader_election.cc:288] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
W20240503 02:13:32.723410 3886 leader_election.cc:341] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.723623 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d1771c9b104e52a24a0411f05f0f94" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.723979 4008 raft_consensus.cc:482] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.724135 4008 raft_consensus.cc:504] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.724602 4105 leader_election.cc:288] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.725317 4008 leader_election.cc:288] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.726226 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a88f9cd3fa40c88cc361f53c906d4e" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.726429 3848 raft_consensus.cc:3037] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.723920 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d1771c9b104e52a24a0411f05f0f94" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.726900 3921 raft_consensus.cc:3037] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.727082 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.727185 4010 raft_consensus.cc:482] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.727303 4010 raft_consensus.cc:504] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.728144 3962 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.728296 3962 ts_tablet_manager.cc:1367] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.129s user 0.010s sys 0.010s
I20240503 02:13:32.728920 4105 raft_consensus.cc:482] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.729035 4105 raft_consensus.cc:504] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.729507 4105 leader_election.cc:288] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.723907 3767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d1771c9b104e52a24a0411f05f0f94" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.731209 4016 consensus_queue.cc:1035] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.730043 3914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1aea07837f41a281d484e37eab5744" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.725834 3917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89de6eace7d643e2a78385a67c771991" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.732511 4008 consensus_queue.cc:1035] T 141b1d61e275407a9827ce8ba0bd1913 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.723695 4108 leader_election.cc:288] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
W20240503 02:13:32.733150 3812 leader_election.cc:341] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.733623 3842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e4e31c162b4f5aabed85d5183c7ad5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.733691 3765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518cefb57371493496e276f5a10fd53d" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.733867 3842 raft_consensus.cc:2454] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.734014 3773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a88f9cd3fa40c88cc361f53c906d4e" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.734124 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d1771c9b104e52a24a0411f05f0f94" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.734179 3766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e4e31c162b4f5aabed85d5183c7ad5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:32.734452 3886 leader_election.cc:341] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.734125 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1aea07837f41a281d484e37eab5744" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.734649 3885 leader_election.cc:302] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:32.734903 4108 raft_consensus.cc:2781] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.734967 4016 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20240503 02:13:32.735061 4108 raft_consensus.cc:482] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.735167 3914 raft_consensus.cc:2454] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:32.734915 4105 consensus_queue.cc:1035] T 33ee6a0a5f584efe81cf64179fd6c4f0 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:13:32.731009 3962 raft_consensus.cc:348] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:32.745414 3812 leader_election.cc:341] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.746160 3886 leader_election.cc:341] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.746404 4108 raft_consensus.cc:3037] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.747707 3962 raft_consensus.cc:374] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.748553 4016 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:13:32.748883 3812 leader_election.cc:341] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.748558 4010 leader_election.cc:288] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
W20240503 02:13:32.749567 3886 leader_election.cc:341] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.749811 3886 leader_election.cc:341] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.749998 3886 leader_election.cc:341] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.750643 3964 tablet_bootstrap.cc:492] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.750847 3964 ts_tablet_manager.cc:1367] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.172s user 0.018s sys 0.018s
I20240503 02:13:32.751855 3964 raft_consensus.cc:348] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.752091 3964 raft_consensus.cc:374] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.752213 3964 raft_consensus.cc:729] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.753108 3967 raft_consensus.cc:504] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.753718 4110 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.754235 3967 leader_election.cc:288] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.755072 3767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9a7019777d47c48621b1b40e7ef540" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.752483 3964 consensus_queue.cc:260] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.760177 3814 leader_election.cc:302] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.760545 3967 raft_consensus.cc:2781] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.766232 3962 raft_consensus.cc:729] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.766980 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5127e67d5abf450eafbf14b22cde7939" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.767385 3766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:32.767570 3886 leader_election.cc:341] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.767678 3812 leader_election.cc:341] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.779469 4108 raft_consensus.cc:504] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.748165 3840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518cefb57371493496e276f5a10fd53d" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.789311 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e4e31c162b4f5aabed85d5183c7ad5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.756294 3843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d1771c9b104e52a24a0411f05f0f94" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.791242 3843 raft_consensus.cc:2379] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4001ccb96b064ad2a2ca9c61096516fe in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:32.791914 3922 raft_consensus.cc:2379] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:32.748620 3845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5127e67d5abf450eafbf14b22cde7939" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
W20240503 02:13:32.793936 3886 leader_election.cc:341] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.794742 3967 raft_consensus.cc:482] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.794803 3840 raft_consensus.cc:2454] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:32.794713 3847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.795133 4108 leader_election.cc:288] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.798789 3885 leader_election.cc:302] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.799927 3847 raft_consensus.cc:2379] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4001ccb96b064ad2a2ca9c61096516fe in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:32.800140 3967 raft_consensus.cc:3037] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.802980 4008 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
W20240503 02:13:32.806334 3814 leader_election.cc:341] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.807813 3964 ts_tablet_manager.cc:1398] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20240503 02:13:32.808215 3964 tablet_bootstrap.cc:492] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.809288 3964 tablet_bootstrap.cc:654] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.809693 3848 raft_consensus.cc:2454] T d3a88f9cd3fa40c88cc361f53c906d4e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.809762 3984 raft_consensus.cc:482] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.809803 4016 raft_consensus.cc:482] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.809916 3984 raft_consensus.cc:504] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.809986 4016 raft_consensus.cc:504] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.810638 3984 leader_election.cc:288] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.810801 4016 leader_election.cc:288] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.811755 4110 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.812279 3814 leader_election.cc:302] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.812867 3962 consensus_queue.cc:260] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.814934 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948ee5eebd9c405fa85066b3a57ebe02" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.817112 3984 raft_consensus.cc:2726] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.817813 3766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e4e31c162b4f5aabed85d5183c7ad5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.818230 3921 raft_consensus.cc:2454] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.818574 3963 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.818738 4091 raft_consensus.cc:482] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.818776 3885 leader_election.cc:302] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.819401 4094 raft_consensus.cc:482] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.818787 3963 ts_tablet_manager.cc:1367] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.120s user 0.006s sys 0.014s
I20240503 02:13:32.820982 3962 ts_tablet_manager.cc:1398] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.087s user 0.001s sys 0.001s
I20240503 02:13:32.821503 3846 raft_consensus.cc:3037] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.818888 4091 raft_consensus.cc:504] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.821851 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:32.822585 4110 raft_consensus.cc:2781] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:32.822711 4016 raft_consensus.cc:482] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.822628 4094 raft_consensus.cc:504] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:32.823010 3886 leader_election.cc:341] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.823341 3885 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.823635 4105 raft_consensus.cc:2726] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.823675 3922 raft_consensus.cc:2379] T 948ee5eebd9c405fa85066b3a57ebe02 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
W20240503 02:13:32.823637 3812 leader_election.cc:341] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.823911 3962 tablet_bootstrap.cc:492] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.823742 4108 raft_consensus.cc:2726] T 73d1771c9b104e52a24a0411f05f0f94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.826455 3967 raft_consensus.cc:504] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.827010 4091 leader_election.cc:288] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.827073 3967 leader_election.cc:288] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
W20240503 02:13:32.824375 3814 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.157.67:34983, user_credentials={real_user=slave}} blocked reactor thread for 46272.4us
I20240503 02:13:32.831748 3814 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.832262 4008 raft_consensus.cc:2726] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.832381 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab19c51fbe2e4417b2fb32f20d51d5bc" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.834648 4110 raft_consensus.cc:482] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:32.834796 4110 raft_consensus.cc:3037] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.836766 4110 raft_consensus.cc:504] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.837354 4110 leader_election.cc:288] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:32.838263 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c32e8b3939447ab864df8945502369" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:32.839743 3812 leader_election.cc:341] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.839814 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.841308 3962 tablet_bootstrap.cc:654] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.841511 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9a7019777d47c48621b1b40e7ef540" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.843513 3768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518cefb57371493496e276f5a10fd53d" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.844652 3846 raft_consensus.cc:2454] T 48e4e31c162b4f5aabed85d5183c7ad5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.844707 3964 tablet_bootstrap.cc:492] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
W20240503 02:13:32.845111 3885 leader_election.cc:341] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.845293 3885 leader_election.cc:302] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.845108 4016 raft_consensus.cc:504] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.846024 3885 leader_election.cc:302] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.845155 3915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab19c51fbe2e4417b2fb32f20d51d5bc" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.846655 3769 raft_consensus.cc:3037] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.847070 3915 raft_consensus.cc:2379] T ab19c51fbe2e4417b2fb32f20d51d5bc P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:32.847684 4108 raft_consensus.cc:2726] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.848140 3814 leader_election.cc:302] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.848649 3984 raft_consensus.cc:2781] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
W20240503 02:13:32.848753 3814 leader_election.cc:341] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.848901 3814 leader_election.cc:302] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.849288 4008 consensus_queue.cc:1035] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20240503 02:13:32.849896 4008 consensus_queue.cc:1035] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.850661 4066 raft_consensus.cc:2726] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.852316 3885 leader_election.cc:302] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.858364 4094 leader_election.cc:288] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.858606 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c32e8b3939447ab864df8945502369" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.859022 3963 raft_consensus.cc:348] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.859350 3963 raft_consensus.cc:374] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.859489 3963 raft_consensus.cc:729] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.859820 3963 consensus_queue.cc:260] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.860673 3963 ts_tablet_manager.cc:1398] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.041s user 0.001s sys 0.001s
I20240503 02:13:32.861078 3963 tablet_bootstrap.cc:492] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.861629 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1aea07837f41a281d484e37eab5744" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:32.861907 3918 raft_consensus.cc:3037] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.863962 3922 raft_consensus.cc:3037] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:32.865998 3886 leader_election.cc:341] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.867058 3964 ts_tablet_manager.cc:1367] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.024s
I20240503 02:13:32.868100 3964 raft_consensus.cc:348] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.868448 3964 raft_consensus.cc:374] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.868571 3964 raft_consensus.cc:729] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.868898 3964 consensus_queue.cc:260] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.869251 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1aea07837f41a281d484e37eab5744" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:32.869702 4110 raft_consensus.cc:2781] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.869890 4110 raft_consensus.cc:686] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
W20240503 02:13:32.869997 3812 leader_election.cc:341] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:32.870060 3915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.870198 4110 consensus_queue.cc:237] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.870507 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518cefb57371493496e276f5a10fd53d" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:32.870736 3846 raft_consensus.cc:3037] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.872287 3984 raft_consensus.cc:686] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.872579 3984 consensus_queue.cc:237] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.872821 4109 consensus_queue.cc:1035] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.873708 3846 raft_consensus.cc:2454] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:32.874105 4108 raft_consensus.cc:2781] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.874286 4108 raft_consensus.cc:686] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.874675 4108 consensus_queue.cc:237] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.875442 3885 leader_election.cc:302] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.875998 4109 raft_consensus.cc:2781] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.876178 4109 raft_consensus.cc:686] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:32.876510 4109 consensus_queue.cc:237] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.877650 3963 tablet_bootstrap.cc:654] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.878477 4056 consensus_queue.cc:1035] T 10fb3ffa08c84ffbbd779d35a68ff7f6 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.878672 4016 leader_election.cc:288] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.847010 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948ee5eebd9c405fa85066b3a57ebe02" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.883937 3964 ts_tablet_manager.cc:1398] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20240503 02:13:32.884423 3964 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.885506 3922 raft_consensus.cc:2454] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.886567 3964 tablet_bootstrap.cc:654] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.887054 3918 raft_consensus.cc:2454] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:32.910620 4056 raft_consensus.cc:482] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:32.910849 4056 raft_consensus.cc:504] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.911702 4056 leader_election.cc:288] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:32.919823 4109 consensus_queue.cc:1035] T 23b76a1a9dea45aebd7fea02fbc9cb42 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.919910 3671 catalog_manager.cc:5484] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.927731 3962 tablet_bootstrap.cc:492] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.927909 3962 ts_tablet_manager.cc:1367] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.107s user 0.017s sys 0.014s
I20240503 02:13:32.928607 3962 raft_consensus.cc:348] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.928783 3962 raft_consensus.cc:374] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.928910 3962 raft_consensus.cc:729] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.929129 3962 consensus_queue.cc:260] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.929760 3962 ts_tablet_manager.cc:1398] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:32.930680 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb8007da5342d29290016341c5999b" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:32.932825 3962 tablet_bootstrap.cc:492] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.934286 4132 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.935045 3962 tablet_bootstrap.cc:654] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.939487 3964 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.939678 3964 ts_tablet_manager.cc:1367] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.000s
I20240503 02:13:32.940613 3964 raft_consensus.cc:348] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.940894 3964 raft_consensus.cc:374] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.941025 3964 raft_consensus.cc:729] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.942171 3964 consensus_queue.cc:260] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.942865 3962 tablet_bootstrap.cc:492] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:32.943261 3668 catalog_manager.cc:5484] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.946041 3962 ts_tablet_manager.cc:1367] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20240503 02:13:32.946306 3668 catalog_manager.cc:5484] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.946707 3963 tablet_bootstrap.cc:492] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.941036 3671 catalog_manager.cc:5484] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:32.946933 3963 ts_tablet_manager.cc:1367] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.086s user 0.004s sys 0.000s
I20240503 02:13:32.947333 3962 raft_consensus.cc:348] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.947531 3962 raft_consensus.cc:374] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.947610 3962 raft_consensus.cc:729] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.947836 3963 raft_consensus.cc:348] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.948174 3963 raft_consensus.cc:374] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.948289 3963 raft_consensus.cc:729] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.943994 3964 ts_tablet_manager.cc:1398] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:32.948609 3963 consensus_queue.cc:260] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.949064 3964 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.949370 3963 ts_tablet_manager.cc:1398] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.949064 4109 consensus_queue.cc:1035] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:13:32.952512 3964 tablet_bootstrap.cc:654] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
W20240503 02:13:32.944607 3814 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.157.67:34983, user_credentials={real_user=slave}} blocked reactor thread for 95858.6us
W20240503 02:13:32.953634 3814 leader_election.cc:341] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:32.944923 3719 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 02:13:32.953830 3814 leader_election.cc:302] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.954080 4008 raft_consensus.cc:2726] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:32.954249 3814 leader_election.cc:302] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.954504 3967 raft_consensus.cc:2781] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.954639 3967 raft_consensus.cc:686] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.954931 3967 consensus_queue.cc:237] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.956583 4010 consensus_queue.cc:1035] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.957751 3963 tablet_bootstrap.cc:492] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.959267 3963 tablet_bootstrap.cc:654] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.962430 3668 catalog_manager.cc:5484] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:32.963762 3964 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:32.963959 3964 ts_tablet_manager.cc:1367] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20240503 02:13:32.964846 3964 raft_consensus.cc:348] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.965124 3964 raft_consensus.cc:374] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.965242 3964 raft_consensus.cc:729] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.965620 3964 consensus_queue.cc:260] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.947865 3962 consensus_queue.cc:260] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.966199 3984 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:32.966459 3964 ts_tablet_manager.cc:1398] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:32.966845 3964 tablet_bootstrap.cc:492] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:32.966320 3962 ts_tablet_manager.cc:1398] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20240503 02:13:32.954643 3814 leader_election.cc:302] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:32.971747 3963 tablet_bootstrap.cc:492] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:32.971906 3963 ts_tablet_manager.cc:1367] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20240503 02:13:32.972692 3963 raft_consensus.cc:348] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.972920 3963 raft_consensus.cc:374] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:32.973076 3963 raft_consensus.cc:729] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:32.973356 3963 consensus_queue.cc:260] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:32.975174 3962 tablet_bootstrap.cc:492] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:32.976534 4134 raft_consensus.cc:2781] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:32.976778 3736 leader_election.cc:302] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.977432 3736 leader_election.cc:302] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:32.977726 3964 tablet_bootstrap.cc:654] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.985857 3963 ts_tablet_manager.cc:1398] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:32.986300 3963 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:32.986979 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353eaefa93a64cd19b50fcfb916bfed6" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.987672 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb8007da5342d29290016341c5999b" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:32.987952 3846 raft_consensus.cc:2379] T 43cb8007da5342d29290016341c5999b P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:32.988692 3963 tablet_bootstrap.cc:654] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:32.989660 3975 raft_consensus.cc:3037] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.990322 4013 raft_consensus.cc:3037] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:32.985649 4134 raft_consensus.cc:686] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:32.996980 4132 consensus_queue.cc:1035] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20240503 02:13:33.004137 3962 tablet_bootstrap.cc:654] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.005616 4094 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20240503 02:13:33.010629 3668 catalog_manager.cc:5484] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.015233 3736 leader_election.cc:302] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.016163 3984 consensus_queue.cc:1035] T 02fb95baa11649fdbf58393d0de53fff P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20240503 02:13:33.016855 3975 raft_consensus.cc:2726] T ab19c51fbe2e4417b2fb32f20d51d5bc P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.018266 3736 leader_election.cc:302] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.019052 4056 raft_consensus.cc:2726] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.019080 4016 raft_consensus.cc:3037] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.029687 4013 raft_consensus.cc:2726] T 948ee5eebd9c405fa85066b3a57ebe02 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.003926 4134 consensus_queue.cc:237] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.033753 3984 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:13:33.035486 3984 consensus_queue.cc:1035] T 948ee5eebd9c405fa85066b3a57ebe02 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:13:33.070011 4132 raft_consensus.cc:482] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.070287 4132 raft_consensus.cc:504] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.071390 4132 leader_election.cc:288] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.071910 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.076853 4016 raft_consensus.cc:2726] T 43cb8007da5342d29290016341c5999b P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240503 02:13:33.078106 3812 leader_election.cc:341] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.087195 4102 consensus_queue.cc:1035] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20240503 02:13:33.089255 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.094822 3814 leader_election.cc:341] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.095053 3814 leader_election.cc:302] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.096760 3963 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.099196 3963 ts_tablet_manager.cc:1367] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.111s user 0.006s sys 0.030s
I20240503 02:13:33.099226 4016 raft_consensus.cc:482] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.099412 4016 raft_consensus.cc:504] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.100152 3963 raft_consensus.cc:348] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.100821 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518cefb57371493496e276f5a10fd53d" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.101176 4132 raft_consensus.cc:2726] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.102162 4016 leader_election.cc:288] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.104701 3963 raft_consensus.cc:374] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.104807 3963 raft_consensus.cc:729] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.105242 3963 consensus_queue.cc:260] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.106181 3963 ts_tablet_manager.cc:1398] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20240503 02:13:33.106647 3963 tablet_bootstrap.cc:492] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.108831 3963 tablet_bootstrap.cc:654] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.109629 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518cefb57371493496e276f5a10fd53d" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.110481 3846 raft_consensus.cc:2379] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:33.123523 4134 consensus_queue.cc:1035] T ab19c51fbe2e4417b2fb32f20d51d5bc P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.126508 3668 catalog_manager.cc:5484] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.127049 3962 tablet_bootstrap.cc:492] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.127190 3962 ts_tablet_manager.cc:1367] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.157s user 0.026s sys 0.019s
I20240503 02:13:33.127842 3962 raft_consensus.cc:348] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.128059 3962 raft_consensus.cc:374] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.128152 3962 raft_consensus.cc:729] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.128352 3962 consensus_queue.cc:260] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.128850 3962 ts_tablet_manager.cc:1398] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:33.129128 3962 tablet_bootstrap.cc:492] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.141649 3736 leader_election.cc:302] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.142067 4016 raft_consensus.cc:3037] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.143334 4110 raft_consensus.cc:482] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.144409 4110 raft_consensus.cc:504] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.144994 4110 leader_election.cc:288] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.145658 3764 raft_consensus.cc:1226] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Rejecting Update request from peer 4001ccb96b064ad2a2ca9c61096516fe for earlier term 1. Current term is 2. Ops: []
I20240503 02:13:33.146050 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.146234 3846 raft_consensus.cc:2454] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.146569 3885 leader_election.cc:302] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.146957 4110 raft_consensus.cc:2781] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.147055 4110 raft_consensus.cc:482] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.147147 4110 raft_consensus.cc:3037] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.147822 4109 consensus_queue.cc:1035] T 43cb8007da5342d29290016341c5999b P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20240503 02:13:33.148330 3964 tablet_bootstrap.cc:492] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.149585 4102 raft_consensus.cc:482] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.149757 4016 raft_consensus.cc:2726] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.150242 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.150188 4102 raft_consensus.cc:504] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:33.150739 3886 leader_election.cc:341] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.150803 4102 leader_election.cc:288] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.151221 4010 consensus_queue.cc:1046] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20240503 02:13:33.151736 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.151947 3846 raft_consensus.cc:2454] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.152774 4010 raft_consensus.cc:3032] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Stepping down as leader of term 1
I20240503 02:13:33.154836 3885 leader_election.cc:302] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.154584 4010 raft_consensus.cc:729] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.155504 4109 raft_consensus.cc:2781] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.155622 4109 raft_consensus.cc:482] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.155890 4109 raft_consensus.cc:3037] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.156822 4110 raft_consensus.cc:504] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.157264 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.157585 4010 consensus_queue.cc:260] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:33.158274 3886 leader_election.cc:341] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.158627 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.159538 3841 raft_consensus.cc:3037] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.162488 4010 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:33.163998 3964 ts_tablet_manager.cc:1367] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.182s user 0.006s sys 0.040s
I20240503 02:13:33.164650 4110 leader_election.cc:288] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.163753 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.166388 3964 raft_consensus.cc:348] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.168817 4109 raft_consensus.cc:504] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:33.169652 3886 leader_election.cc:341] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.170552 3964 raft_consensus.cc:374] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.171258 4109 leader_election.cc:288] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.171738 3962 tablet_bootstrap.cc:654] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.171303 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.172394 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.172391 3844 raft_consensus.cc:3037] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.173851 3841 raft_consensus.cc:2454] T b04a17e29c044fcaab0eb6e836eca141 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
W20240503 02:13:33.175787 3886 leader_election.cc:341] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.176409 3964 raft_consensus.cc:729] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.176635 3964 consensus_queue.cc:260] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.175446 3885 leader_election.cc:302] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.178136 4010 raft_consensus.cc:2781] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.179395 3963 tablet_bootstrap.cc:492] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.180797 4010 raft_consensus.cc:686] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.181154 4010 consensus_queue.cc:237] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.182545 3963 ts_tablet_manager.cc:1367] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.073s user 0.014s sys 0.009s
I20240503 02:13:33.183458 3844 raft_consensus.cc:2454] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.183683 3964 ts_tablet_manager.cc:1398] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20240503 02:13:33.183748 3962 tablet_bootstrap.cc:492] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.183985 3964 tablet_bootstrap.cc:492] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.184262 3885 leader_election.cc:302] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.184724 4109 raft_consensus.cc:2781] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.184926 4134 raft_consensus.cc:482] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.185070 4134 raft_consensus.cc:504] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.186245 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34af2cfd01c7406da5ed8da1ce0cf129" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.186288 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34af2cfd01c7406da5ed8da1ce0cf129" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.186529 3962 ts_tablet_manager.cc:1367] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.000s
W20240503 02:13:33.186717 3812 leader_election.cc:341] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.187304 3962 raft_consensus.cc:348] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.187551 3962 raft_consensus.cc:374] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.187637 3962 raft_consensus.cc:729] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.187903 3962 consensus_queue.cc:260] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.188233 4109 raft_consensus.cc:686] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.188519 4109 consensus_queue.cc:237] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.189565 4134 leader_election.cc:288] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.189834 3963 raft_consensus.cc:348] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.189963 3920 raft_consensus.cc:2454] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.190080 3963 raft_consensus.cc:374] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.190197 3963 raft_consensus.cc:729] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.190848 3964 tablet_bootstrap.cc:654] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.190855 3814 leader_election.cc:302] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.191560 3963 consensus_queue.cc:260] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.193408 3962 ts_tablet_manager.cc:1398] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20240503 02:13:33.194252 4134 raft_consensus.cc:2781] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.198746 3963 ts_tablet_manager.cc:1398] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20240503 02:13:33.199115 3963 tablet_bootstrap.cc:492] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.205791 3963 tablet_bootstrap.cc:654] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.205932 4134 raft_consensus.cc:482] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.206202 4134 raft_consensus.cc:3037] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.206948 3962 tablet_bootstrap.cc:492] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.209127 4134 raft_consensus.cc:504] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.209363 3670 catalog_manager.cc:5484] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.210443 3962 tablet_bootstrap.cc:654] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.210522 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34af2cfd01c7406da5ed8da1ce0cf129" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.211031 3964 tablet_bootstrap.cc:492] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
W20240503 02:13:33.211059 3812 leader_election.cc:341] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.212772 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34af2cfd01c7406da5ed8da1ce0cf129" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.213044 4134 leader_election.cc:288] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.213650 3963 tablet_bootstrap.cc:492] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.214099 3920 raft_consensus.cc:3037] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.214242 3964 ts_tablet_manager.cc:1367] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.009s
I20240503 02:13:33.214634 3963 ts_tablet_manager.cc:1367] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.001s
I20240503 02:13:33.215178 3963 raft_consensus.cc:348] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.215071 3964 raft_consensus.cc:348] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.215485 3963 raft_consensus.cc:374] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.215670 3963 raft_consensus.cc:729] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.215693 3964 raft_consensus.cc:374] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.215965 3964 raft_consensus.cc:729] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.217346 3920 raft_consensus.cc:2454] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.217743 3814 leader_election.cc:302] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.221633 3962 tablet_bootstrap.cc:492] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.221771 4134 raft_consensus.cc:2781] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.221714 3963 consensus_queue.cc:260] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.221926 4134 raft_consensus.cc:686] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:33.221983 3962 ts_tablet_manager.cc:1367] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.003s
I20240503 02:13:33.222206 4134 consensus_queue.cc:237] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.222414 3963 ts_tablet_manager.cc:1398] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:13:33.222754 3963 tablet_bootstrap.cc:492] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.223148 3962 raft_consensus.cc:348] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.223456 3962 raft_consensus.cc:374] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.223623 3962 raft_consensus.cc:729] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.223935 3962 consensus_queue.cc:260] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.224665 3962 ts_tablet_manager.cc:1398] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:33.224749 3963 tablet_bootstrap.cc:654] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.225097 3962 tablet_bootstrap.cc:492] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.225144 4094 raft_consensus.cc:482] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.225286 4094 raft_consensus.cc:504] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.225996 4094 leader_election.cc:288] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.226752 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d23a9965894fc2aed70be32e716613" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.227177 3962 tablet_bootstrap.cc:654] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.227432 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d23a9965894fc2aed70be32e716613" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.227581 3814 leader_election.cc:341] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:33.228260 3812 leader_election.cc:341] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.228397 3812 leader_election.cc:302] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.228893 4094 raft_consensus.cc:2726] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.229859 3964 consensus_queue.cc:260] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.230571 3964 ts_tablet_manager.cc:1398] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20240503 02:13:33.229362 3670 catalog_manager.cc:5484] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.230832 3671 catalog_manager.cc:5484] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.231969 3964 tablet_bootstrap.cc:492] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.233808 3964 tablet_bootstrap.cc:654] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.248363 3964 tablet_bootstrap.cc:492] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.248539 3964 ts_tablet_manager.cc:1367] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.003s
I20240503 02:13:33.248574 3963 tablet_bootstrap.cc:492] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.248929 3962 tablet_bootstrap.cc:492] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.249138 3962 ts_tablet_manager.cc:1367] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.003s
I20240503 02:13:33.249184 3963 ts_tablet_manager.cc:1367] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.016s
I20240503 02:13:33.249486 3964 raft_consensus.cc:348] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.249783 3964 raft_consensus.cc:374] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.249907 3964 raft_consensus.cc:729] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.249841 3962 raft_consensus.cc:348] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.250056 3962 raft_consensus.cc:374] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.250164 3962 raft_consensus.cc:729] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.250160 4016 raft_consensus.cc:482] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.250353 4016 raft_consensus.cc:504] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.250442 3962 consensus_queue.cc:260] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.250638 3963 raft_consensus.cc:348] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.251034 4016 leader_election.cc:288] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.251077 3962 ts_tablet_manager.cc:1398] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.251780 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1aea07837f41a281d484e37eab5744" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.251988 3963 raft_consensus.cc:374] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.251372 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1aea07837f41a281d484e37eab5744" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.252358 3963 raft_consensus.cc:729] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.252033 3920 raft_consensus.cc:2379] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.252637 3963 consensus_queue.cc:260] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.253118 3739 leader_election.cc:302] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.253470 3963 ts_tablet_manager.cc:1398] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:33.253707 4016 raft_consensus.cc:3037] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.253916 3963 tablet_bootstrap.cc:492] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.254335 4010 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.255795 3963 tablet_bootstrap.cc:654] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.256296 4016 raft_consensus.cc:2726] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.251338 3964 consensus_queue.cc:260] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.252218 3962 tablet_bootstrap.cc:492] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.258107 3964 ts_tablet_manager.cc:1398] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20240503 02:13:33.259408 3962 tablet_bootstrap.cc:654] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.259667 4016 raft_consensus.cc:482] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.259807 4016 raft_consensus.cc:504] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.260672 4016 leader_election.cc:288] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.261777 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9a7019777d47c48621b1b40e7ef540" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.261965 3920 raft_consensus.cc:2379] T de9a7019777d47c48621b1b40e7ef540 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.264801 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9a7019777d47c48621b1b40e7ef540" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.265247 3736 leader_election.cc:302] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.265738 3975 raft_consensus.cc:3037] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.268427 3975 raft_consensus.cc:2726] T de9a7019777d47c48621b1b40e7ef540 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.272714 3964 tablet_bootstrap.cc:492] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.274323 3964 tablet_bootstrap.cc:654] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.275067 3963 tablet_bootstrap.cc:492] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.275255 3963 ts_tablet_manager.cc:1367] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.009s
I20240503 02:13:33.276162 3962 tablet_bootstrap.cc:492] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.276376 3962 ts_tablet_manager.cc:1367] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.006s
I20240503 02:13:33.277230 3962 raft_consensus.cc:348] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.277580 3962 raft_consensus.cc:374] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.277694 3962 raft_consensus.cc:729] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.278014 3962 consensus_queue.cc:260] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.278755 3962 ts_tablet_manager.cc:1398] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:33.279072 3962 tablet_bootstrap.cc:492] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.281025 3963 raft_consensus.cc:348] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.281298 3963 raft_consensus.cc:374] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.281368 3963 raft_consensus.cc:729] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.281571 4110 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.281667 3963 consensus_queue.cc:260] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.282471 3962 tablet_bootstrap.cc:654] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.286577 3963 ts_tablet_manager.cc:1398] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20240503 02:13:33.288333 3963 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.290144 3963 tablet_bootstrap.cc:654] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.292613 3964 tablet_bootstrap.cc:492] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.292742 3964 ts_tablet_manager.cc:1367] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20240503 02:13:33.293437 3964 raft_consensus.cc:348] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.293640 3964 raft_consensus.cc:374] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.293700 3964 raft_consensus.cc:729] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.294553 3964 consensus_queue.cc:260] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.295056 3963 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.295202 3963 ts_tablet_manager.cc:1367] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20240503 02:13:33.295280 3964 ts_tablet_manager.cc:1398] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.295931 3963 raft_consensus.cc:348] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.296134 3963 raft_consensus.cc:374] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.296186 3963 raft_consensus.cc:729] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.296397 3963 consensus_queue.cc:260] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.297046 3963 ts_tablet_manager.cc:1398] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:33.297331 3963 tablet_bootstrap.cc:492] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.298341 3962 tablet_bootstrap.cc:492] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.298471 3962 ts_tablet_manager.cc:1367] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.002s
I20240503 02:13:33.299305 3963 tablet_bootstrap.cc:654] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.299209 3962 raft_consensus.cc:348] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.299407 3962 raft_consensus.cc:374] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.299458 3962 raft_consensus.cc:729] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.299659 3962 consensus_queue.cc:260] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.300298 3962 ts_tablet_manager.cc:1398] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:33.300577 3962 tablet_bootstrap.cc:492] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.300720 3964 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.302052 3964 tablet_bootstrap.cc:654] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.303442 3962 tablet_bootstrap.cc:654] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.307062 3984 consensus_queue.cc:1035] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:13:33.307590 3886 consensus_peers.cc:485] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Couldn't send request to peer 6e0d1128f0404d609df72fd607c20650. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:33.309559 3964 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.309684 3964 ts_tablet_manager.cc:1367] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20240503 02:13:33.310370 3964 raft_consensus.cc:348] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.310598 3964 raft_consensus.cc:374] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.310655 3964 raft_consensus.cc:729] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.310865 3964 consensus_queue.cc:260] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.312726 3964 ts_tablet_manager.cc:1398] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:33.313004 3964 tablet_bootstrap.cc:492] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.317307 3964 tablet_bootstrap.cc:654] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.318801 3771 raft_consensus.cc:3037] T 73d1771c9b104e52a24a0411f05f0f94 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.325874 3962 tablet_bootstrap.cc:492] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.326081 3962 ts_tablet_manager.cc:1367] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.003s
I20240503 02:13:33.326341 3964 tablet_bootstrap.cc:492] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.326506 3964 ts_tablet_manager.cc:1367] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240503 02:13:33.327231 3962 raft_consensus.cc:348] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.327297 3964 raft_consensus.cc:348] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.327528 3962 raft_consensus.cc:374] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.327556 3964 raft_consensus.cc:374] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.327735 3962 raft_consensus.cc:729] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.327769 3964 raft_consensus.cc:729] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.328092 3962 consensus_queue.cc:260] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.328099 3964 consensus_queue.cc:260] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.328722 3962 ts_tablet_manager.cc:1398] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.328816 3964 ts_tablet_manager.cc:1398] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.329141 3964 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.330902 4010 raft_consensus.cc:482] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.331066 3964 tablet_bootstrap.cc:654] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.331050 4010 raft_consensus.cc:504] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.331686 4010 leader_election.cc:288] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.332175 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c32e8b3939447ab864df8945502369" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.332340 3844 raft_consensus.cc:2454] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.332755 3885 leader_election.cc:302] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.333709 3967 consensus_queue.cc:1035] T 73d1771c9b104e52a24a0411f05f0f94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.334579 3962 tablet_bootstrap.cc:492] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.335675 4110 raft_consensus.cc:2781] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.335767 4110 raft_consensus.cc:482] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.335844 4110 raft_consensus.cc:3037] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.336266 3962 tablet_bootstrap.cc:654] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.339169 4110 raft_consensus.cc:504] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.340505 3768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c32e8b3939447ab864df8945502369" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.340673 3768 raft_consensus.cc:3037] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.341512 4010 consensus_queue.cc:1035] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.343595 3768 raft_consensus.cc:2454] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.344116 3886 leader_election.cc:302] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.344527 4010 raft_consensus.cc:2781] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.344635 4010 raft_consensus.cc:686] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.344870 4010 consensus_queue.cc:237] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.345285 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c32e8b3939447ab864df8945502369" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.345436 3846 raft_consensus.cc:3037] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.348038 3964 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.348214 3964 ts_tablet_manager.cc:1367] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.002s
I20240503 02:13:33.348727 3846 raft_consensus.cc:2454] T 89c32e8b3939447ab864df8945502369 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.349172 4134 raft_consensus.cc:482] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.349341 4134 raft_consensus.cc:504] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.350142 4134 leader_election.cc:288] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.350713 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5127e67d5abf450eafbf14b22cde7939" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.350962 3764 raft_consensus.cc:2454] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.351061 4110 leader_election.cc:288] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.351383 3812 leader_election.cc:302] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.352020 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5127e67d5abf450eafbf14b22cde7939" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.352218 3920 raft_consensus.cc:2454] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.353453 3963 tablet_bootstrap.cc:492] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.353626 3963 ts_tablet_manager.cc:1367] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.056s user 0.006s sys 0.009s
I20240503 02:13:33.354326 3963 raft_consensus.cc:348] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.354535 3963 raft_consensus.cc:374] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.354614 3963 raft_consensus.cc:729] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.354833 3963 consensus_queue.cc:260] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.355384 3963 ts_tablet_manager.cc:1398] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:33.355681 3963 tablet_bootstrap.cc:492] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.355908 3962 tablet_bootstrap.cc:492] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.356021 3962 ts_tablet_manager.cc:1367] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.007s
I20240503 02:13:33.356606 3962 raft_consensus.cc:348] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.356789 3962 raft_consensus.cc:374] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.356853 3962 raft_consensus.cc:729] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.357014 3962 consensus_queue.cc:260] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.357304 3967 raft_consensus.cc:2781] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.350752 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c32e8b3939447ab864df8945502369" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.357369 3963 tablet_bootstrap.cc:654] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.357376 3967 raft_consensus.cc:482] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.360486 3967 raft_consensus.cc:3037] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.363178 3967 raft_consensus.cc:504] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.363729 3967 leader_election.cc:288] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.357393 3764 raft_consensus.cc:3037] T 48e4e31c162b4f5aabed85d5183c7ad5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.349319 3964 raft_consensus.cc:348] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.364235 3964 raft_consensus.cc:374] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.364333 3964 raft_consensus.cc:729] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.364490 3768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5127e67d5abf450eafbf14b22cde7939" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.359236 3771 raft_consensus.cc:2362] T 89c32e8b3939447ab864df8945502369 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1. Re-sending same reply.
I20240503 02:13:33.364651 3768 raft_consensus.cc:3037] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.364634 3964 consensus_queue.cc:260] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.367172 3964 ts_tablet_manager.cc:1398] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.019s user 0.001s sys 0.002s
I20240503 02:13:33.367530 3964 tablet_bootstrap.cc:492] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.368207 3768 raft_consensus.cc:2454] T 5127e67d5abf450eafbf14b22cde7939 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.367956 3670 catalog_manager.cc:5484] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.369000 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5127e67d5abf450eafbf14b22cde7939" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.369144 3920 raft_consensus.cc:3037] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.369596 3964 tablet_bootstrap.cc:654] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.371702 3920 raft_consensus.cc:2454] T 5127e67d5abf450eafbf14b22cde7939 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.372097 3814 leader_election.cc:302] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.372473 4134 raft_consensus.cc:2781] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.372570 4134 raft_consensus.cc:686] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:33.372772 4134 consensus_queue.cc:237] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.375406 4010 consensus_queue.cc:1035] T 48e4e31c162b4f5aabed85d5183c7ad5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.357692 3962 ts_tablet_manager.cc:1398] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:33.377231 3962 tablet_bootstrap.cc:492] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.377441 4110 consensus_queue.cc:1035] T d3a88f9cd3fa40c88cc361f53c906d4e P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.378876 3964 tablet_bootstrap.cc:492] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.378978 3962 tablet_bootstrap.cc:654] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.379046 3964 ts_tablet_manager.cc:1367] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20240503 02:13:33.380926 3963 tablet_bootstrap.cc:492] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.381103 3963 ts_tablet_manager.cc:1367] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.006s
I20240503 02:13:33.381973 3963 raft_consensus.cc:348] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.382211 3963 raft_consensus.cc:374] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.382302 3963 raft_consensus.cc:729] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.382565 3963 consensus_queue.cc:260] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.383139 3963 ts_tablet_manager.cc:1398] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.383508 3967 raft_consensus.cc:482] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.383649 3967 raft_consensus.cc:504] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.384433 3967 leader_election.cc:288] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.385046 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2758e6b755a9470e872f11fe2fb9001a" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.385524 3814 leader_election.cc:341] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.385867 3768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2758e6b755a9470e872f11fe2fb9001a" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.386303 3812 leader_election.cc:341] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.386361 3963 tablet_bootstrap.cc:492] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.386404 3812 leader_election.cc:302] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.386821 4066 raft_consensus.cc:2726] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.388597 3963 tablet_bootstrap.cc:654] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.389982 3964 raft_consensus.cc:348] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.390234 3964 raft_consensus.cc:374] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.390313 3964 raft_consensus.cc:729] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.390591 3964 consensus_queue.cc:260] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.391278 3964 ts_tablet_manager.cc:1398] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20240503 02:13:33.391620 3964 tablet_bootstrap.cc:492] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.393692 3964 tablet_bootstrap.cc:654] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.397977 3962 tablet_bootstrap.cc:492] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.398200 3962 ts_tablet_manager.cc:1367] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.000s
I20240503 02:13:33.397977 3670 catalog_manager.cc:5484] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.398999 3962 raft_consensus.cc:348] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.400550 3962 raft_consensus.cc:374] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.400681 3962 raft_consensus.cc:729] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.400913 3962 consensus_queue.cc:260] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.401605 3962 ts_tablet_manager.cc:1398] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:33.401906 3962 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.402463 3964 tablet_bootstrap.cc:492] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.402638 3964 ts_tablet_manager.cc:1367] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20240503 02:13:33.403472 3964 raft_consensus.cc:348] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.403712 3964 raft_consensus.cc:374] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.403786 3964 raft_consensus.cc:729] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.403997 3962 tablet_bootstrap.cc:654] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.404072 3964 consensus_queue.cc:260] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.407289 3964 ts_tablet_manager.cc:1398] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20240503 02:13:33.407625 3964 tablet_bootstrap.cc:492] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.410380 3964 tablet_bootstrap.cc:654] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.412644 3963 tablet_bootstrap.cc:492] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.412799 3963 ts_tablet_manager.cc:1367] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.008s
I20240503 02:13:33.413753 4110 raft_consensus.cc:482] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.413911 4110 raft_consensus.cc:504] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.414535 4110 leader_election.cc:288] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.415446 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
W20240503 02:13:33.416002 3885 leader_election.cc:341] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.416241 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.416107 3963 raft_consensus.cc:348] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.416417 3963 raft_consensus.cc:374] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.416494 3963 raft_consensus.cc:729] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
W20240503 02:13:33.416649 3886 leader_election.cc:341] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.416764 3886 leader_election.cc:302] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.416725 3963 consensus_queue.cc:260] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.417200 4105 raft_consensus.cc:2726] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.418133 3962 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.418251 3962 ts_tablet_manager.cc:1367] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.005s
I20240503 02:13:33.418980 3962 raft_consensus.cc:348] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.419169 3962 raft_consensus.cc:374] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.419241 3962 raft_consensus.cc:729] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.419504 3963 ts_tablet_manager.cc:1398] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20240503 02:13:33.419467 3962 consensus_queue.cc:260] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.419847 3963 tablet_bootstrap.cc:492] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.421679 4134 raft_consensus.cc:482] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.421808 4134 raft_consensus.cc:504] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.421905 3963 tablet_bootstrap.cc:654] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.422425 4134 leader_election.cc:288] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.423605 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793fe86859754f3d90919e150e2f3693" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.423811 3920 raft_consensus.cc:2454] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.424432 3814 leader_election.cc:302] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.425170 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793fe86859754f3d90919e150e2f3693" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.425647 3812 leader_election.cc:341] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.425941 4134 raft_consensus.cc:2781] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.426018 4134 raft_consensus.cc:482] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.426095 4134 raft_consensus.cc:3037] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.427902 4134 raft_consensus.cc:504] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.428413 4134 leader_election.cc:288] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.428781 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793fe86859754f3d90919e150e2f3693" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.429015 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793fe86859754f3d90919e150e2f3693" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.429167 3920 raft_consensus.cc:3037] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:33.429198 3812 leader_election.cc:341] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.431223 3975 raft_consensus.cc:482] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.431344 3975 raft_consensus.cc:504] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.431933 3975 leader_election.cc:288] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.432262 3920 raft_consensus.cc:2454] T 793fe86859754f3d90919e150e2f3693 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.432303 4110 raft_consensus.cc:482] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.432389 4110 raft_consensus.cc:504] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.432585 3814 leader_election.cc:302] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.432940 4110 leader_election.cc:288] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.433390 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.433571 3844 raft_consensus.cc:2454] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.433665 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.433943 3885 leader_election.cc:302] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
W20240503 02:13:33.434193 3886 leader_election.cc:341] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.434219 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.434391 3846 raft_consensus.cc:2379] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:33.434823 3915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3450807dfc52460dbcbbd75dc8b71cb9" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.435264 4110 raft_consensus.cc:2781] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.435343 4110 raft_consensus.cc:482] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.435423 4110 raft_consensus.cc:3037] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.437306 4110 raft_consensus.cc:504] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.437922 3736 leader_election.cc:302] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.438423 3975 raft_consensus.cc:3037] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.438972 3964 tablet_bootstrap.cc:492] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.439136 3964 ts_tablet_manager.cc:1367] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.007s
I20240503 02:13:33.439461 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.439626 3846 raft_consensus.cc:3037] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.440120 3964 raft_consensus.cc:348] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.440342 3964 raft_consensus.cc:374] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.440407 3964 raft_consensus.cc:729] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.440697 3964 consensus_queue.cc:260] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.441229 3975 raft_consensus.cc:2726] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.441601 4134 raft_consensus.cc:2781] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.441716 4134 raft_consensus.cc:686] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:33.441952 4134 consensus_queue.cc:237] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.443580 3962 ts_tablet_manager.cc:1398] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.025s user 0.001s sys 0.001s
I20240503 02:13:33.444000 3962 tablet_bootstrap.cc:492] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.444733 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:33.448648 3886 leader_election.cc:341] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.446133 3964 ts_tablet_manager.cc:1398] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20240503 02:13:33.446709 3846 raft_consensus.cc:2454] T a5e9aab321bd48c38b354316119370d2 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.449797 3885 leader_election.cc:302] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.450073 4105 raft_consensus.cc:2781] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.450212 4105 raft_consensus.cc:686] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.450443 4105 consensus_queue.cc:237] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.450973 3962 tablet_bootstrap.cc:654] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.446848 4110 leader_election.cc:288] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.449215 3670 catalog_manager.cc:5484] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.452832 3964 tablet_bootstrap.cc:492] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.460948 3668 catalog_manager.cc:5484] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.462615 3963 tablet_bootstrap.cc:492] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.462725 3963 ts_tablet_manager.cc:1367] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.043s user 0.012s sys 0.004s
I20240503 02:13:33.464082 3963 raft_consensus.cc:348] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.464613 3963 raft_consensus.cc:374] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.464708 3963 raft_consensus.cc:729] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.465499 3963 consensus_queue.cc:260] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.466504 3963 ts_tablet_manager.cc:1398] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:33.467013 3963 tablet_bootstrap.cc:492] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.469826 3964 tablet_bootstrap.cc:654] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.469848 3962 tablet_bootstrap.cc:492] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.469960 3962 ts_tablet_manager.cc:1367] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.007s
I20240503 02:13:33.470844 3962 raft_consensus.cc:348] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.471092 3962 raft_consensus.cc:374] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.471159 3962 raft_consensus.cc:729] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.471374 3962 consensus_queue.cc:260] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.471815 3963 tablet_bootstrap.cc:654] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.472131 3962 ts_tablet_manager.cc:1398] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.472393 3962 tablet_bootstrap.cc:492] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.474325 3962 tablet_bootstrap.cc:654] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.484012 3964 tablet_bootstrap.cc:492] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.484231 3964 ts_tablet_manager.cc:1367] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.000s
I20240503 02:13:33.485244 3962 tablet_bootstrap.cc:492] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.485379 3962 ts_tablet_manager.cc:1367] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20240503 02:13:33.485774 3963 tablet_bootstrap.cc:492] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.485885 3963 ts_tablet_manager.cc:1367] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.001s
I20240503 02:13:33.486114 3962 raft_consensus.cc:348] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.486325 3962 raft_consensus.cc:374] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.486424 3962 raft_consensus.cc:729] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.486580 3963 raft_consensus.cc:348] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.486626 3962 consensus_queue.cc:260] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.486759 3963 raft_consensus.cc:374] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.486811 3963 raft_consensus.cc:729] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.487020 3963 consensus_queue.cc:260] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.487627 3963 ts_tablet_manager.cc:1398] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.488590 3962 ts_tablet_manager.cc:1398] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240503 02:13:33.488843 3962 tablet_bootstrap.cc:492] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.490547 3962 tablet_bootstrap.cc:654] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.486110 3964 raft_consensus.cc:348] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.491361 3964 raft_consensus.cc:374] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.491433 3964 raft_consensus.cc:729] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.491667 3964 consensus_queue.cc:260] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.492480 3964 ts_tablet_manager.cc:1398] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20240503 02:13:33.492861 3964 tablet_bootstrap.cc:492] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.493608 3962 tablet_bootstrap.cc:492] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.493846 3962 ts_tablet_manager.cc:1367] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20240503 02:13:33.494475 3964 tablet_bootstrap.cc:654] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.494782 3962 raft_consensus.cc:348] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.495100 3962 raft_consensus.cc:374] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.495203 3962 raft_consensus.cc:729] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.495463 3962 consensus_queue.cc:260] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.496198 3962 ts_tablet_manager.cc:1398] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:33.496536 3962 tablet_bootstrap.cc:492] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.498632 3962 tablet_bootstrap.cc:654] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.501377 3975 raft_consensus.cc:482] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.501545 3975 raft_consensus.cc:504] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.502375 3975 leader_election.cc:288] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.502571 4134 raft_consensus.cc:482] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.502704 4134 raft_consensus.cc:504] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.503371 4134 leader_election.cc:288] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.504141 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34af2cfd01c7406da5ed8da1ce0cf129" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.505116 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.505877 4105 raft_consensus.cc:482] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.506042 4105 raft_consensus.cc:504] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.506806 4105 leader_election.cc:288] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.507372 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d23a9965894fc2aed70be32e716613" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.508080 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d23a9965894fc2aed70be32e716613" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.508325 3846 raft_consensus.cc:2454] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
W20240503 02:13:33.508847 3886 leader_election.cc:341] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.509114 3885 leader_election.cc:302] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.509622 4105 raft_consensus.cc:2781] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.509752 4105 raft_consensus.cc:482] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.509841 4105 raft_consensus.cc:3037] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:33.513722 3812 leader_election.cc:341] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.514402 4105 raft_consensus.cc:504] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.514916 3962 tablet_bootstrap.cc:492] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.515115 3962 ts_tablet_manager.cc:1367] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.008s
I20240503 02:13:33.515540 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d23a9965894fc2aed70be32e716613" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:33.516127 3886 leader_election.cc:341] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.516150 3962 raft_consensus.cc:348] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.516485 3962 raft_consensus.cc:374] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.516610 3962 raft_consensus.cc:729] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.516878 3962 consensus_queue.cc:260] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.517045 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d23a9965894fc2aed70be32e716613" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.517254 3846 raft_consensus.cc:3037] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.517911 3962 ts_tablet_manager.cc:1398] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:33.518363 3962 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.520431 3962 tablet_bootstrap.cc:654] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.521292 3846 raft_consensus.cc:2454] T e8d23a9965894fc2aed70be32e716613 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.521780 3975 raft_consensus.cc:482] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.521958 3975 raft_consensus.cc:504] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.522672 3975 leader_election.cc:288] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.523304 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31d78763218f4ea3af2c4f4620e3cde4" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.523558 3844 raft_consensus.cc:2454] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:33.523979 3736 leader_election.cc:302] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.524866 3885 leader_election.cc:302] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.525189 4110 raft_consensus.cc:2781] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.525365 4110 raft_consensus.cc:686] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.525701 4110 consensus_queue.cc:237] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.526916 3975 raft_consensus.cc:2781] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.527032 3975 raft_consensus.cc:482] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.527118 3975 raft_consensus.cc:3037] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.532904 3975 raft_consensus.cc:504] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.534068 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31d78763218f4ea3af2c4f4620e3cde4" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.534269 3844 raft_consensus.cc:3037] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.539884 3844 raft_consensus.cc:2454] T 31d78763218f4ea3af2c4f4620e3cde4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:33.540338 3736 leader_election.cc:302] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.540644 4016 raft_consensus.cc:2781] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.540668 3975 leader_election.cc:288] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.540813 4016 raft_consensus.cc:686] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:33.541090 4016 consensus_queue.cc:237] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.541610 4105 leader_election.cc:288] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.549144 3671 catalog_manager.cc:5484] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.551517 3962 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.551728 3962 ts_tablet_manager.cc:1367] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.015s
I20240503 02:13:33.552667 3962 raft_consensus.cc:348] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.552953 3962 raft_consensus.cc:374] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.553058 3962 raft_consensus.cc:729] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.553354 3962 consensus_queue.cc:260] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.553609 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31d78763218f4ea3af2c4f4620e3cde4" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.553745 3918 raft_consensus.cc:2454] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:33.554188 3962 ts_tablet_manager.cc:1398] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.554667 3962 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.555883 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31d78763218f4ea3af2c4f4620e3cde4" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.556147 3919 raft_consensus.cc:3037] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.556931 3962 tablet_bootstrap.cc:654] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.558391 4066 consensus_queue.cc:1035] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.553450 3915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34af2cfd01c7406da5ed8da1ce0cf129" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.559506 3915 raft_consensus.cc:2379] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.553479 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.561295 3814 leader_election.cc:341] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.561398 3814 leader_election.cc:302] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.561983 4094 raft_consensus.cc:2726] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.562150 4094 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.562937 3739 leader_election.cc:302] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.563290 4016 raft_consensus.cc:3037] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.568866 3671 catalog_manager.cc:5484] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.574110 4105 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.576138 3919 raft_consensus.cc:2454] T 31d78763218f4ea3af2c4f4620e3cde4 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:33.580013 4016 raft_consensus.cc:2726] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.580588 3967 consensus_queue.cc:1035] T de9a7019777d47c48621b1b40e7ef540 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.582013 3967 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.585552 4110 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.597790 4016 raft_consensus.cc:482] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.597970 4016 raft_consensus.cc:504] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.599031 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a88f9cd3fa40c88cc361f53c906d4e" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.600272 4016 leader_election.cc:288] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.600630 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a88f9cd3fa40c88cc361f53c906d4e" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.601287 3739 leader_election.cc:302] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.601953 4016 raft_consensus.cc:2726] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.609001 3963 tablet_bootstrap.cc:492] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.611505 3963 tablet_bootstrap.cc:654] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.615823 3962 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.615869 3963 tablet_bootstrap.cc:492] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.615933 3964 tablet_bootstrap.cc:492] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.616101 3963 ts_tablet_manager.cc:1367] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20240503 02:13:33.616143 3962 ts_tablet_manager.cc:1367] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.062s user 0.028s sys 0.015s
I20240503 02:13:33.616278 3964 ts_tablet_manager.cc:1367] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.123s user 0.052s sys 0.051s
I20240503 02:13:33.616950 3962 raft_consensus.cc:348] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.616976 3963 raft_consensus.cc:348] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.617286 3963 raft_consensus.cc:374] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.617314 3962 raft_consensus.cc:374] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.617404 3963 raft_consensus.cc:729] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.617463 3962 raft_consensus.cc:729] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.617475 3964 raft_consensus.cc:348] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.617740 3963 consensus_queue.cc:260] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.617790 3962 consensus_queue.cc:260] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.617954 3964 raft_consensus.cc:374] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.621560 3964 raft_consensus.cc:729] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.621892 3964 consensus_queue.cc:260] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.622572 3964 ts_tablet_manager.cc:1398] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20240503 02:13:33.618515 3962 ts_tablet_manager.cc:1398] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.619300 3963 ts_tablet_manager.cc:1398] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:13:33.622983 3962 tablet_bootstrap.cc:492] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.623147 3963 tablet_bootstrap.cc:492] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.624202 3964 tablet_bootstrap.cc:492] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.624920 3963 tablet_bootstrap.cc:654] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.626407 3964 tablet_bootstrap.cc:654] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.626893 3962 tablet_bootstrap.cc:654] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.631659 4110 raft_consensus.cc:482] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.631865 4110 raft_consensus.cc:504] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.632658 4110 leader_election.cc:288] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.633285 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15eeedfc81114776a8614759c7741de9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.633545 3846 raft_consensus.cc:2454] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.634114 3885 leader_election.cc:302] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.634797 4108 raft_consensus.cc:2781] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.634938 4108 raft_consensus.cc:482] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.635049 4108 raft_consensus.cc:3037] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.638032 3768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15eeedfc81114776a8614759c7741de9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.638739 3886 leader_election.cc:341] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.640508 4108 raft_consensus.cc:504] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.642642 4110 raft_consensus.cc:482] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.642838 4110 raft_consensus.cc:504] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.643674 4110 leader_election.cc:288] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.644308 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d0bfa5dbdc44928bed580d8198659c" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.644562 3844 raft_consensus.cc:2454] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.645100 3768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15eeedfc81114776a8614759c7741de9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.645160 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15eeedfc81114776a8614759c7741de9" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.645399 3846 raft_consensus.cc:3037] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:33.645749 3886 leader_election.cc:341] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.646211 4066 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.646773 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d0bfa5dbdc44928bed580d8198659c" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.646987 3764 raft_consensus.cc:2454] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.647890 3885 leader_election.cc:302] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.649565 4110 raft_consensus.cc:2781] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.649714 4110 raft_consensus.cc:482] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.649828 4110 raft_consensus.cc:3037] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.653100 4066 raft_consensus.cc:482] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.653273 4066 raft_consensus.cc:504] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.654068 4066 leader_election.cc:288] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.654719 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072c60ef4387498abec90d6716f856b8" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.655253 3814 leader_election.cc:341] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.655884 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072c60ef4387498abec90d6716f856b8" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.656342 3812 leader_election.cc:341] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.656447 3812 leader_election.cc:302] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.656885 4066 raft_consensus.cc:2726] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.653357 4110 raft_consensus.cc:504] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.659018 4010 raft_consensus.cc:482] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.661772 4108 leader_election.cc:288] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.659174 4010 raft_consensus.cc:504] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.662619 4010 leader_election.cc:288] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.663115 4110 leader_election.cc:288] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.664137 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d0bfa5dbdc44928bed580d8198659c" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.664350 3841 raft_consensus.cc:3037] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.667820 3841 raft_consensus.cc:2454] T 01d0bfa5dbdc44928bed580d8198659c P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.668165 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965bbbd3d2e74bdc81d09b292e5ed8c5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.668407 3844 raft_consensus.cc:2454] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.670589 3885 leader_election.cc:302] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.670909 3885 leader_election.cc:302] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.671172 4010 raft_consensus.cc:482] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.671300 4010 raft_consensus.cc:504] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.671967 4010 leader_election.cc:288] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.672206 4010 raft_consensus.cc:2781] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.672324 4010 raft_consensus.cc:686] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.672526 4105 raft_consensus.cc:2781] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.672616 4105 raft_consensus.cc:482] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.672601 4010 consensus_queue.cc:237] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.672899 4105 raft_consensus.cc:3037] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.674453 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d0bfa5dbdc44928bed580d8198659c" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.674657 3771 raft_consensus.cc:3037] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.676064 4105 raft_consensus.cc:504] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.676669 4105 leader_election.cc:288] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.677366 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965bbbd3d2e74bdc81d09b292e5ed8c5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.677704 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89de6eace7d643e2a78385a67c771991" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.677892 3769 raft_consensus.cc:2454] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.678402 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965bbbd3d2e74bdc81d09b292e5ed8c5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.678941 3770 raft_consensus.cc:3037] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.679459 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89de6eace7d643e2a78385a67c771991" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.679677 3844 raft_consensus.cc:2454] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.680393 4134 raft_consensus.cc:482] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.681456 4134 raft_consensus.cc:504] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.682153 4134 leader_election.cc:288] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.682543 3770 raft_consensus.cc:2454] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.683427 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ff048917b34a4b9cd09dee267fc078" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.684381 3814 leader_election.cc:341] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.684581 4134 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.685974 4108 raft_consensus.cc:482] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.686105 4108 raft_consensus.cc:504] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.686569 3885 leader_election.cc:302] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.686865 4109 raft_consensus.cc:2781] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.686934 4109 raft_consensus.cc:482] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.686991 4109 raft_consensus.cc:3037] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.689350 3771 raft_consensus.cc:2454] T 01d0bfa5dbdc44928bed580d8198659c P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.689749 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965bbbd3d2e74bdc81d09b292e5ed8c5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.690016 3841 raft_consensus.cc:3037] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.691166 3846 raft_consensus.cc:2454] T 15eeedfc81114776a8614759c7741de9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.691578 3963 tablet_bootstrap.cc:492] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.691582 3885 leader_election.cc:302] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.691771 3963 ts_tablet_manager.cc:1367] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.069s user 0.006s sys 0.025s
I20240503 02:13:33.692055 4105 raft_consensus.cc:2781] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.692250 4105 raft_consensus.cc:686] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.692538 3963 raft_consensus.cc:348] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.692737 3963 raft_consensus.cc:374] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.692579 4105 consensus_queue.cc:237] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.692791 3963 raft_consensus.cc:729] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.693040 3963 consensus_queue.cc:260] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.693568 3963 ts_tablet_manager.cc:1398] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.693821 3963 tablet_bootstrap.cc:492] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.701848 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd36fffa3034c6cbe511f4f39b078ff" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.702082 3846 raft_consensus.cc:2454] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.702483 3885 leader_election.cc:302] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.698326 4108 leader_election.cc:288] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.702080 4109 raft_consensus.cc:504] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.703361 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ff048917b34a4b9cd09dee267fc078" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.703881 3812 leader_election.cc:341] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.703979 3812 leader_election.cc:302] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.704128 3963 tablet_bootstrap.cc:654] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.704187 4066 raft_consensus.cc:2726] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.704484 4108 raft_consensus.cc:2781] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.704530 3886 leader_election.cc:302] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.704588 4108 raft_consensus.cc:482] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.704676 4108 raft_consensus.cc:3037] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.704944 4105 raft_consensus.cc:2781] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.705066 4105 raft_consensus.cc:686] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.705493 4109 leader_election.cc:288] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.705363 4105 consensus_queue.cc:237] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.705930 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89de6eace7d643e2a78385a67c771991" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.706043 3846 raft_consensus.cc:3037] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.705739 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89de6eace7d643e2a78385a67c771991" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.706712 3771 raft_consensus.cc:3037] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.708369 3846 raft_consensus.cc:2454] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.708842 3771 raft_consensus.cc:2454] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.709100 3885 leader_election.cc:302] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.709337 4110 raft_consensus.cc:2781] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.709427 4110 raft_consensus.cc:686] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.709410 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd36fffa3034c6cbe511f4f39b078ff" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.709822 4110 consensus_queue.cc:237] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.710629 3962 tablet_bootstrap.cc:492] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
W20240503 02:13:33.710789 3886 leader_election.cc:341] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.711688 3962 ts_tablet_manager.cc:1367] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.089s user 0.017s sys 0.022s
I20240503 02:13:33.719714 3841 raft_consensus.cc:2454] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.722543 3964 tablet_bootstrap.cc:492] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.722699 3964 ts_tablet_manager.cc:1367] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.099s user 0.021s sys 0.025s
I20240503 02:13:33.723582 4066 raft_consensus.cc:482] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.723745 4066 raft_consensus.cc:504] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.724550 4066 leader_election.cc:288] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.725558 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.725790 3921 raft_consensus.cc:2454] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.725841 4066 raft_consensus.cc:482] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.725950 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.731176 3814 leader_election.cc:302] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.732704 4134 raft_consensus.cc:2781] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.732811 4134 raft_consensus.cc:482] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.732903 4134 raft_consensus.cc:3037] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.733208 4016 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.733340 4016 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:33.733618 3812 leader_election.cc:341] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.730424 4066 raft_consensus.cc:504] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.739068 4066 leader_election.cc:288] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.746466 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.746793 3841 raft_consensus.cc:2379] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.746924 4016 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.748772 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df08ada758d5442cba4f7bd85b445352" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.749485 3812 leader_election.cc:341] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.751000 4108 raft_consensus.cc:504] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.752308 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd36fffa3034c6cbe511f4f39b078ff" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.752513 3841 raft_consensus.cc:3037] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.753597 3962 raft_consensus.cc:348] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.753772 3962 raft_consensus.cc:374] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.753842 3962 raft_consensus.cc:729] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.754026 3962 consensus_queue.cc:260] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.751822 4108 leader_election.cc:288] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.756057 3841 raft_consensus.cc:2454] T ddd36fffa3034c6cbe511f4f39b078ff P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.756279 3964 raft_consensus.cc:348] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.756443 3885 leader_election.cc:302] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:33.756309 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd36fffa3034c6cbe511f4f39b078ff" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:33.756989 3886 leader_election.cc:341] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.757570 4010 raft_consensus.cc:2781] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.757628 3962 ts_tablet_manager.cc:1398] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.046s user 0.002s sys 0.000s
I20240503 02:13:33.756449 3964 raft_consensus.cc:374] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.757879 3964 raft_consensus.cc:729] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.758152 3962 tablet_bootstrap.cc:492] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.757686 4010 raft_consensus.cc:686] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.758610 4010 consensus_queue.cc:237] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.758936 4108 consensus_queue.cc:1035] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.759217 3964 consensus_queue.cc:260] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.761718 3963 tablet_bootstrap.cc:492] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.763128 3963 ts_tablet_manager.cc:1367] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.069s user 0.004s sys 0.003s
I20240503 02:13:33.762065 3964 ts_tablet_manager.cc:1398] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.039s user 0.001s sys 0.001s
I20240503 02:13:33.763945 3964 tablet_bootstrap.cc:492] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.764166 3962 tablet_bootstrap.cc:654] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.767956 4109 consensus_queue.cc:1035] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.767319 4102 consensus_queue.cc:1035] T 89c32e8b3939447ab864df8945502369 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:13:33.769814 4134 raft_consensus.cc:504] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.770416 4134 leader_election.cc:288] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.771950 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.772455 4134 raft_consensus.cc:482] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240503 02:13:33.772464 3812 leader_election.cc:341] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.772589 4134 raft_consensus.cc:504] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.773878 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da09c39261443bbf898a390937ffb4" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.774521 3812 leader_election.cc:341] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.780320 4134 leader_election.cc:288] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.780615 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.780794 3886 consensus_peers.cc:485] T b04a17e29c044fcaab0eb6e836eca141 P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Couldn't send request to peer 6e0d1128f0404d609df72fd607c20650. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:33.780831 3921 raft_consensus.cc:2379] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:33.781414 3739 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.781829 4016 raft_consensus.cc:3037] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.781952 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da09c39261443bbf898a390937ffb4" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.782274 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.782433 3921 raft_consensus.cc:3037] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.784541 4016 raft_consensus.cc:2726] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.784981 3921 raft_consensus.cc:2454] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.785634 3814 leader_election.cc:302] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.785992 4066 raft_consensus.cc:2781] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.786136 4066 raft_consensus.cc:686] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:33.786455 4066 consensus_queue.cc:237] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.787763 3963 raft_consensus.cc:348] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.788311 3963 raft_consensus.cc:374] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.788385 3963 raft_consensus.cc:729] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.788573 3963 consensus_queue.cc:260] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.789263 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df08ada758d5442cba4f7bd85b445352" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.789808 3814 leader_election.cc:341] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.789913 3814 leader_election.cc:302] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.790177 3920 raft_consensus.cc:2454] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.791321 3964 tablet_bootstrap.cc:654] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.791694 3963 ts_tablet_manager.cc:1398] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.028s user 0.000s sys 0.002s
I20240503 02:13:33.791967 3963 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.792439 4134 raft_consensus.cc:2726] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.793001 3814 leader_election.cc:302] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.793380 4134 raft_consensus.cc:2781] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.793483 4134 raft_consensus.cc:482] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.793601 4134 raft_consensus.cc:3037] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.809005 4134 raft_consensus.cc:504] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.809897 3671 catalog_manager.cc:5484] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.810657 3671 catalog_manager.cc:5484] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.811375 3984 consensus_queue.cc:1035] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.812393 4134 leader_election.cc:288] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.812945 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da09c39261443bbf898a390937ffb4" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.813128 3919 raft_consensus.cc:3037] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.813580 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da09c39261443bbf898a390937ffb4" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.813688 3770 raft_consensus.cc:3037] T d3a88f9cd3fa40c88cc361f53c906d4e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:13:33.814061 3812 leader_election.cc:341] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.811254 3671 catalog_manager.cc:5484] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.814533 3671 catalog_manager.cc:5484] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.816648 4134 raft_consensus.cc:482] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.816761 4134 raft_consensus.cc:504] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.820261 4134 leader_election.cc:288] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.820482 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4972943b854b28a0c927ce1a0511d7" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.820776 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4972943b854b28a0c927ce1a0511d7" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.820897 3812 leader_election.cc:341] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.822012 4016 raft_consensus.cc:482] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.822166 4016 raft_consensus.cc:504] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.823732 4134 raft_consensus.cc:482] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.823870 4134 raft_consensus.cc:504] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:33.824564 3814 leader_election.cc:341] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.824703 3814 leader_election.cc:302] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.825016 4016 leader_election.cc:288] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.825305 3984 raft_consensus.cc:2726] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.825366 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.826433 3964 tablet_bootstrap.cc:492] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.826570 3964 ts_tablet_manager.cc:1367] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.000s
I20240503 02:13:33.827229 4134 leader_election.cc:288] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.828299 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.828497 3921 raft_consensus.cc:2454] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:33.828838 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.829041 3771 raft_consensus.cc:2454] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.829377 3812 leader_election.cc:302] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.829653 4134 raft_consensus.cc:2781] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.829723 4134 raft_consensus.cc:482] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.829802 4134 raft_consensus.cc:3037] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.830626 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.830814 3921 raft_consensus.cc:2454] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:33.831442 3739 leader_election.cc:302] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.832630 4016 raft_consensus.cc:2781] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.832742 4016 raft_consensus.cc:482] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.832831 4016 raft_consensus.cc:3037] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.834301 3984 raft_consensus.cc:482] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.834484 3984 raft_consensus.cc:504] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.835275 3984 leader_election.cc:288] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.836773 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6622d0a3f54d418d588fe0281d9324" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.837407 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6622d0a3f54d418d588fe0281d9324" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:33.837832 3814 leader_election.cc:341] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:13:33.838479 3812 leader_election.cc:341] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.838631 3812 leader_election.cc:302] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.839118 3984 raft_consensus.cc:2726] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.841245 3919 raft_consensus.cc:2454] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.841785 3814 leader_election.cc:302] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.842255 3984 raft_consensus.cc:2781] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.842387 3984 raft_consensus.cc:686] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:33.842693 3984 consensus_queue.cc:237] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.844476 3964 raft_consensus.cc:348] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.844714 3964 raft_consensus.cc:374] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.844786 3964 raft_consensus.cc:729] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.847218 3963 tablet_bootstrap.cc:654] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.845453 3964 consensus_queue.cc:260] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.849397 4134 raft_consensus.cc:504] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.849705 3846 raft_consensus.cc:2379] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.849967 4134 leader_election.cc:288] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.850385 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.850869 3962 tablet_bootstrap.cc:492] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.850991 3962 ts_tablet_manager.cc:1367] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.016s
I20240503 02:13:33.851222 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.851356 3919 raft_consensus.cc:3037] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.851735 3964 ts_tablet_manager.cc:1398] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20240503 02:13:33.851995 3964 tablet_bootstrap.cc:492] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.858238 4016 raft_consensus.cc:504] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.858551 3771 raft_consensus.cc:2379] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in current term 1: Already voted for candidate 6e0d1128f0404d609df72fd607c20650 in this term.
I20240503 02:13:33.858698 4016 leader_election.cc:288] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.859133 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.859308 3846 raft_consensus.cc:2379] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.859860 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea39b9e4e11247ab95cc690e60b22b11" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:33.857604 3967 consensus_queue.cc:1035] T 5127e67d5abf450eafbf14b22cde7939 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.857506 3962 raft_consensus.cc:348] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.860461 3962 raft_consensus.cc:374] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.860545 3962 raft_consensus.cc:729] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.860728 3962 consensus_queue.cc:260] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.861101 3962 ts_tablet_manager.cc:1398] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20240503 02:13:33.862844 3739 leader_election.cc:302] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.863286 4016 raft_consensus.cc:2726] T ea39b9e4e11247ab95cc690e60b22b11 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.865465 3962 tablet_bootstrap.cc:492] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.867244 4238 raft_consensus.cc:482] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.867372 4238 raft_consensus.cc:504] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.867972 4238 leader_election.cc:288] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.868505 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2758e6b755a9470e872f11fe2fb9001a" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.868626 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2758e6b755a9470e872f11fe2fb9001a" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.868871 3846 raft_consensus.cc:2454] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
W20240503 02:13:33.868984 3886 leader_election.cc:341] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.875980 3885 leader_election.cc:302] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.876830 3963 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.876960 3964 tablet_bootstrap.cc:654] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.877089 3963 ts_tablet_manager.cc:1367] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.085s user 0.004s sys 0.006s
I20240503 02:13:33.877147 4238 raft_consensus.cc:2781] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.877241 4238 raft_consensus.cc:482] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.877318 4238 raft_consensus.cc:3037] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.877758 3919 raft_consensus.cc:2454] T ea39b9e4e11247ab95cc690e60b22b11 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:33.878104 3814 leader_election.cc:302] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.878584 3984 raft_consensus.cc:2781] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.878890 3984 raft_consensus.cc:686] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:33.879065 3984 consensus_queue.cc:237] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.879665 4016 raft_consensus.cc:482] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.879786 4016 raft_consensus.cc:504] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.880223 4016 leader_election.cc:288] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.880587 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da09c39261443bbf898a390937ffb4" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.880795 3919 raft_consensus.cc:2379] T d9da09c39261443bbf898a390937ffb4 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:33.881431 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da09c39261443bbf898a390937ffb4" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.881879 3736 leader_election.cc:302] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:33.882086 4016 raft_consensus.cc:3037] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.882722 3668 catalog_manager.cc:5484] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.884362 4016 raft_consensus.cc:2726] T d9da09c39261443bbf898a390937ffb4 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.884147 4238 raft_consensus.cc:504] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.885640 4238 leader_election.cc:288] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.886014 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2758e6b755a9470e872f11fe2fb9001a" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.886147 3846 raft_consensus.cc:3037] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.887053 3963 raft_consensus.cc:348] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.887313 3963 raft_consensus.cc:374] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.887398 3963 raft_consensus.cc:729] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.887660 3963 consensus_queue.cc:260] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.887724 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2758e6b755a9470e872f11fe2fb9001a" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:33.888280 3886 leader_election.cc:341] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.889087 3967 consensus_queue.cc:1035] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.892087 3846 raft_consensus.cc:2454] T 2758e6b755a9470e872f11fe2fb9001a P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.892570 3885 leader_election.cc:302] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.892959 4110 raft_consensus.cc:2781] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.888705 3671 catalog_manager.cc:5484] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.893131 4110 raft_consensus.cc:686] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.893414 4110 consensus_queue.cc:237] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.897135 3963 ts_tablet_manager.cc:1398] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20240503 02:13:33.897476 3963 tablet_bootstrap.cc:492] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.904474 3963 tablet_bootstrap.cc:654] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.906080 3962 tablet_bootstrap.cc:654] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.907855 3964 tablet_bootstrap.cc:492] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.908043 3964 ts_tablet_manager.cc:1367] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.004s
I20240503 02:13:33.909090 3964 raft_consensus.cc:348] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.909369 3964 raft_consensus.cc:374] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.909438 3964 raft_consensus.cc:729] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.909587 3668 catalog_manager.cc:5484] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.909759 3964 consensus_queue.cc:260] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.910212 3668 catalog_manager.cc:5484] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:33.910682 3964 ts_tablet_manager.cc:1398] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:33.911051 3964 tablet_bootstrap.cc:492] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.911597 4110 consensus_queue.cc:1035] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.913312 3964 tablet_bootstrap.cc:654] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.916344 3963 tablet_bootstrap.cc:492] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.916488 3963 ts_tablet_manager.cc:1367] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.001s
I20240503 02:13:33.917359 3963 raft_consensus.cc:348] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.917626 3963 raft_consensus.cc:374] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.917693 3963 raft_consensus.cc:729] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.917930 3963 consensus_queue.cc:260] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.918501 3963 ts_tablet_manager.cc:1398] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.918768 3963 tablet_bootstrap.cc:492] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
W20240503 02:13:33.920334 3886 consensus_peers.cc:485] T a5e9aab321bd48c38b354316119370d2 P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Couldn't send request to peer 6e0d1128f0404d609df72fd607c20650. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20240503 02:13:33.921892 3812 consensus_peers.cc:485] T 793fe86859754f3d90919e150e2f3693 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Couldn't send request to peer 6e0d1128f0404d609df72fd607c20650. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:33.924063 3963 tablet_bootstrap.cc:654] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.925951 3668 catalog_manager.cc:5484] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:33.927265 3962 tablet_bootstrap.cc:492] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.927474 3962 ts_tablet_manager.cc:1367] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.062s user 0.004s sys 0.002s
I20240503 02:13:33.928040 3964 tablet_bootstrap.cc:492] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.928583 3964 ts_tablet_manager.cc:1367] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.001s
I20240503 02:13:33.928668 3963 tablet_bootstrap.cc:492] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.928786 3963 ts_tablet_manager.cc:1367] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20240503 02:13:33.929219 3962 raft_consensus.cc:348] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.929289 3963 raft_consensus.cc:348] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.929466 3963 raft_consensus.cc:374] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.929486 3962 raft_consensus.cc:374] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.929435 3964 raft_consensus.cc:348] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.929571 3963 raft_consensus.cc:729] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.929647 3962 raft_consensus.cc:729] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.929689 3964 raft_consensus.cc:374] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.929899 3964 raft_consensus.cc:729] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.930133 3964 consensus_queue.cc:260] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.930127 3962 consensus_queue.cc:260] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.930799 3964 ts_tablet_manager.cc:1398] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:33.930954 3962 ts_tablet_manager.cc:1398] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:33.931277 3962 tablet_bootstrap.cc:492] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.931350 3964 tablet_bootstrap.cc:492] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.933169 3962 tablet_bootstrap.cc:654] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.933193 3964 tablet_bootstrap.cc:654] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.933933 4016 consensus_queue.cc:1035] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:33.929905 3963 consensus_queue.cc:260] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.937347 3963 ts_tablet_manager.cc:1398] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20240503 02:13:33.937692 3963 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.939006 3964 tablet_bootstrap.cc:492] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.939179 3964 ts_tablet_manager.cc:1367] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20240503 02:13:33.939603 3963 tablet_bootstrap.cc:654] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.940128 3964 raft_consensus.cc:348] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.940382 3964 raft_consensus.cc:374] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.940516 3964 raft_consensus.cc:729] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.940788 3964 consensus_queue.cc:260] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.943126 3964 ts_tablet_manager.cc:1398] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:33.943423 4110 raft_consensus.cc:482] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.943526 3964 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:33.943630 4110 raft_consensus.cc:504] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.944442 4110 leader_election.cc:288] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.945197 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6622d0a3f54d418d588fe0281d9324" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.946171 3964 tablet_bootstrap.cc:654] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.946267 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6622d0a3f54d418d588fe0281d9324" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.945492 3841 raft_consensus.cc:2454] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:33.947072 3885 leader_election.cc:302] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
W20240503 02:13:33.947543 3886 leader_election.cc:341] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.947997 4110 raft_consensus.cc:2781] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.948166 4110 raft_consensus.cc:482] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.948386 4110 raft_consensus.cc:3037] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.948801 3962 tablet_bootstrap.cc:492] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:33.949438 3962 ts_tablet_manager.cc:1367] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.014s
I20240503 02:13:33.953773 4016 raft_consensus.cc:482] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.954638 4110 raft_consensus.cc:504] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.955647 4016 raft_consensus.cc:504] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.956584 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6622d0a3f54d418d588fe0281d9324" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:33.957031 4110 leader_election.cc:288] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:33.957455 3963 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.957230 3962 raft_consensus.cc:348] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.957824 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6622d0a3f54d418d588fe0281d9324" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:33.961819 3846 raft_consensus.cc:3037] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.963630 3963 ts_tablet_manager.cc:1367] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.001s
I20240503 02:13:33.968364 3964 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:33.969034 3962 raft_consensus.cc:374] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 02:13:33.969502 3886 leader_election.cc:341] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.971244 3846 raft_consensus.cc:2454] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:33.972110 3962 raft_consensus.cc:729] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.972127 3975 consensus_queue.cc:1035] T 31d78763218f4ea3af2c4f4620e3cde4 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20240503 02:13:33.973004 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ff048917b34a4b9cd09dee267fc078" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:33.974387 3885 leader_election.cc:302] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.975689 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ff048917b34a4b9cd09dee267fc078" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:33.976326 3844 raft_consensus.cc:2454] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:33.977581 4110 raft_consensus.cc:2781] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:33.978525 4016 leader_election.cc:288] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.978757 3964 ts_tablet_manager.cc:1367] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.004s
I20240503 02:13:33.978787 3962 consensus_queue.cc:260] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.979976 3920 raft_consensus.cc:2454] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:33.980350 3963 raft_consensus.cc:348] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.980612 3963 raft_consensus.cc:374] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.980741 3963 raft_consensus.cc:729] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.981025 3963 consensus_queue.cc:260] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
W20240503 02:13:33.981326 3719 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20240503 02:13:33.981762 3963 ts_tablet_manager.cc:1398] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20240503 02:13:33.982108 3963 tablet_bootstrap.cc:492] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:33.983498 3963 tablet_bootstrap.cc:654] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:33.984690 3736 leader_election.cc:302] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:33.984752 4110 raft_consensus.cc:686] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:33.984977 4013 raft_consensus.cc:2781] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:33.985173 4013 raft_consensus.cc:482] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:33.985148 4110 consensus_queue.cc:237] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.986012 3962 ts_tablet_manager.cc:1398] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.029s user 0.001s sys 0.001s
I20240503 02:13:33.987854 4013 raft_consensus.cc:3037] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:33.987833 3964 raft_consensus.cc:348] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.992024 3962 tablet_bootstrap.cc:492] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:33.993218 3963 tablet_bootstrap.cc:492] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:33.993661 3984 raft_consensus.cc:482] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:33.993880 3984 raft_consensus.cc:504] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.994707 3984 leader_election.cc:288] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:33.995054 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc392ed1bce416c8e0d47af8c9787bf" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:33.995515 3814 leader_election.cc:341] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.996197 3964 raft_consensus.cc:374] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:33.996541 3964 raft_consensus.cc:729] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:33.996910 3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc392ed1bce416c8e0d47af8c9787bf" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:33.997706 3964 consensus_queue.cc:260] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:33.998414 3963 ts_tablet_manager.cc:1367] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20240503 02:13:33.998621 3964 ts_tablet_manager.cc:1398] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
W20240503 02:13:33.998659 3812 leader_election.cc:341] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:33.998801 3812 leader_election.cc:302] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:33.999770 3984 raft_consensus.cc:2726] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:33.999863 3964 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.000562 3962 tablet_bootstrap.cc:654] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.002499 3964 tablet_bootstrap.cc:654] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.006103 3963 raft_consensus.cc:348] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.008334 4013 raft_consensus.cc:504] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.008838 4013 leader_election.cc:288] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.009217 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ff048917b34a4b9cd09dee267fc078" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.009229 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ff048917b34a4b9cd09dee267fc078" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.009438 3846 raft_consensus.cc:3037] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.009490 3920 raft_consensus.cc:3037] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.011278 3920 raft_consensus.cc:2454] T f6ff048917b34a4b9cd09dee267fc078 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.011636 3739 leader_election.cc:302] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.011880 4013 raft_consensus.cc:2781] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.012054 4013 raft_consensus.cc:686] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:34.012308 4013 consensus_queue.cc:237] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.012626 3846 raft_consensus.cc:2454] T f6ff048917b34a4b9cd09dee267fc078 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.014993 3669 catalog_manager.cc:5484] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.016669 3671 catalog_manager.cc:5484] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.017391 3964 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.017585 3964 ts_tablet_manager.cc:1367] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20240503 02:13:34.017616 3963 raft_consensus.cc:374] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.017702 3963 raft_consensus.cc:729] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.017938 3963 consensus_queue.cc:260] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.018533 3963 ts_tablet_manager.cc:1398] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20240503 02:13:34.018898 3963 tablet_bootstrap.cc:492] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.020388 3962 tablet_bootstrap.cc:492] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:34.020408 3964 raft_consensus.cc:348] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.020565 3962 ts_tablet_manager.cc:1367] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.003s
I20240503 02:13:34.020615 3963 tablet_bootstrap.cc:654] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.020673 3964 raft_consensus.cc:374] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.020936 3964 raft_consensus.cc:729] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.021241 3964 consensus_queue.cc:260] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.021458 3962 raft_consensus.cc:348] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.021698 3962 raft_consensus.cc:374] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.025691 3962 raft_consensus.cc:729] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.025756 3964 ts_tablet_manager.cc:1398] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:34.026167 3962 consensus_queue.cc:260] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.026752 3962 ts_tablet_manager.cc:1398] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20240503 02:13:34.027092 3962 tablet_bootstrap.cc:492] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:34.027827 3964 tablet_bootstrap.cc:492] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.029070 3962 tablet_bootstrap.cc:654] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.029476 3964 tablet_bootstrap.cc:654] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.036082 3764 raft_consensus.cc:3037] T e8d23a9965894fc2aed70be32e716613 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.036943 3963 tablet_bootstrap.cc:492] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.037092 3963 ts_tablet_manager.cc:1367] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.006s
I20240503 02:13:34.037742 3963 raft_consensus.cc:348] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.037957 3963 raft_consensus.cc:374] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.038060 3963 raft_consensus.cc:729] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.038285 3963 consensus_queue.cc:260] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.039412 4110 consensus_queue.cc:1035] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.039857 3963 ts_tablet_manager.cc:1398] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:34.040170 3963 tablet_bootstrap.cc:492] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.045171 3962 tablet_bootstrap.cc:492] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:34.049624 3962 ts_tablet_manager.cc:1367] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.008s
I20240503 02:13:34.051286 3962 raft_consensus.cc:348] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.050138 4109 consensus_queue.cc:1035] T e8d23a9965894fc2aed70be32e716613 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.051715 3962 raft_consensus.cc:374] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.051862 3962 raft_consensus.cc:729] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.050472 3964 tablet_bootstrap.cc:492] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.049818 3963 tablet_bootstrap.cc:654] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.053823 3962 consensus_queue.cc:260] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.054216 3984 raft_consensus.cc:482] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.054409 3984 raft_consensus.cc:504] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.055042 3962 ts_tablet_manager.cc:1398] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240503 02:13:34.055218 3984 leader_election.cc:288] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.055547 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef59225fda142b99782f8e09d65cb9e" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.055580 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef59225fda142b99782f8e09d65cb9e" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.054175 3964 ts_tablet_manager.cc:1367] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.007s
I20240503 02:13:34.056363 3920 raft_consensus.cc:2454] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:34.056854 3814 leader_election.cc:302] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.057050 3964 raft_consensus.cc:348] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.057600 3984 raft_consensus.cc:2781] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.057613 3964 raft_consensus.cc:374] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.057729 3984 raft_consensus.cc:482] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.057750 3964 raft_consensus.cc:729] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.057833 3984 raft_consensus.cc:3037] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.058032 3964 consensus_queue.cc:260] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.058826 3964 ts_tablet_manager.cc:1398] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:34.059226 3964 tablet_bootstrap.cc:492] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
W20240503 02:13:34.057395 3812 leader_election.cc:341] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.060846 3964 tablet_bootstrap.cc:654] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.060909 4109 raft_consensus.cc:482] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.061146 4109 raft_consensus.cc:504] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.061748 4109 leader_election.cc:288] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.062275 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc392ed1bce416c8e0d47af8c9787bf" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.062476 3846 raft_consensus.cc:2454] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.062865 3885 leader_election.cc:302] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.063162 4109 raft_consensus.cc:2781] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.063272 4109 raft_consensus.cc:482] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.063391 4109 raft_consensus.cc:3037] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.063846 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc392ed1bce416c8e0d47af8c9787bf" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:34.064294 3886 leader_election.cc:341] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.065500 3984 raft_consensus.cc:504] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.065914 4109 raft_consensus.cc:504] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.066613 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef59225fda142b99782f8e09d65cb9e" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.066776 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc392ed1bce416c8e0d47af8c9787bf" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:34.066890 3984 leader_election.cc:288] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.066802 3920 raft_consensus.cc:3037] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.067453 3962 tablet_bootstrap.cc:492] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7: Bootstrap starting.
I20240503 02:13:34.068190 4109 leader_election.cc:288] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
W20240503 02:13:34.070771 3886 leader_election.cc:341] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.071343 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef59225fda142b99782f8e09d65cb9e" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:34.071928 3812 leader_election.cc:341] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.072393 3962 tablet_bootstrap.cc:654] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.072728 3920 raft_consensus.cc:2454] T eef59225fda142b99782f8e09d65cb9e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:34.073113 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bc392ed1bce416c8e0d47af8c9787bf" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.073207 3814 leader_election.cc:302] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.073333 3844 raft_consensus.cc:3037] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.073935 4066 raft_consensus.cc:2781] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.074095 4066 raft_consensus.cc:686] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:34.074398 4066 consensus_queue.cc:237] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.077446 4109 consensus_queue.cc:1035] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.080538 4013 raft_consensus.cc:482] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.080760 4013 raft_consensus.cc:504] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.081496 4013 leader_election.cc:288] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.082095 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072c60ef4387498abec90d6716f856b8" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.082329 3846 raft_consensus.cc:2454] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:34.083081 3736 leader_election.cc:302] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.083750 4013 raft_consensus.cc:2781] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.083881 4013 raft_consensus.cc:482] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.083997 4013 raft_consensus.cc:3037] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.087267 4013 raft_consensus.cc:504] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.088304 3844 raft_consensus.cc:2454] T 4bc392ed1bce416c8e0d47af8c9787bf P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.088474 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072c60ef4387498abec90d6716f856b8" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.088718 3846 raft_consensus.cc:3037] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.088811 3885 leader_election.cc:302] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.089360 4109 raft_consensus.cc:2781] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.089550 4109 raft_consensus.cc:686] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:34.089905 4109 consensus_queue.cc:237] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.092226 3846 raft_consensus.cc:2454] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.092842 4013 leader_election.cc:288] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.093079 3736 leader_election.cc:302] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.093158 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072c60ef4387498abec90d6716f856b8" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.093322 3919 raft_consensus.cc:3037] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.093726 4013 raft_consensus.cc:2781] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.093886 4013 raft_consensus.cc:686] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:34.094777 3671 catalog_manager.cc:5484] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.096506 3919 raft_consensus.cc:2454] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.093310 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072c60ef4387498abec90d6716f856b8" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.100433 3920 raft_consensus.cc:2362] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1. Re-sending same reply.
I20240503 02:13:34.103235 4238 consensus_queue.cc:1035] T 01d0bfa5dbdc44928bed580d8198659c P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.109562 3668 catalog_manager.cc:5484] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.109760 4013 consensus_queue.cc:237] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.111614 3964 tablet_bootstrap.cc:492] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.111749 3964 ts_tablet_manager.cc:1367] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.020s
I20240503 02:13:34.112442 3964 raft_consensus.cc:348] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.112604 3964 raft_consensus.cc:374] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.112654 3964 raft_consensus.cc:729] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.112816 3964 consensus_queue.cc:260] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.113263 3964 ts_tablet_manager.cc:1398] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240503 02:13:34.113444 3964 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.115250 3964 tablet_bootstrap.cc:654] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.119948 4238 consensus_queue.cc:1035] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.122579 4238 consensus_queue.cc:1035] T 965bbbd3d2e74bdc81d09b292e5ed8c5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.125191 3964 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.125357 3964 ts_tablet_manager.cc:1367] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20240503 02:13:34.126107 3964 raft_consensus.cc:348] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.126274 3964 raft_consensus.cc:374] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.126324 3964 raft_consensus.cc:729] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.126531 3964 consensus_queue.cc:260] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.127184 3964 ts_tablet_manager.cc:1398] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:34.127425 3964 tablet_bootstrap.cc:492] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.127828 3671 catalog_manager.cc:5484] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.129873 3964 tablet_bootstrap.cc:654] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.135202 4109 consensus_queue.cc:1035] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.138006 3964 tablet_bootstrap.cc:492] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.138164 3964 ts_tablet_manager.cc:1367] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20240503 02:13:34.138789 3964 raft_consensus.cc:348] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.138972 3964 raft_consensus.cc:374] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.139015 3964 raft_consensus.cc:729] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.139163 3964 consensus_queue.cc:260] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.139639 3964 ts_tablet_manager.cc:1398] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:13:34.139883 3964 tablet_bootstrap.cc:492] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.145711 3964 tablet_bootstrap.cc:654] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.147181 3964 tablet_bootstrap.cc:492] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.147281 3964 ts_tablet_manager.cc:1367] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240503 02:13:34.147987 3964 raft_consensus.cc:348] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.148190 3964 raft_consensus.cc:374] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.148252 3964 raft_consensus.cc:729] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.148478 3963 tablet_bootstrap.cc:492] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.148437 3964 consensus_queue.cc:260] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.148581 3963 ts_tablet_manager.cc:1367] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.109s user 0.003s sys 0.005s
I20240503 02:13:34.149022 3964 ts_tablet_manager.cc:1398] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:34.149353 3964 tablet_bootstrap.cc:492] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.150853 3964 tablet_bootstrap.cc:654] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.153694 4013 raft_consensus.cc:482] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.153901 4013 raft_consensus.cc:504] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.154670 4013 leader_election.cc:288] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.155545 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793fe86859754f3d90919e150e2f3693" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.158627 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793fe86859754f3d90919e150e2f3693" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.159237 3736 leader_election.cc:302] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.159787 4013 raft_consensus.cc:2726] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.162357 3963 raft_consensus.cc:348] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.162636 3963 raft_consensus.cc:374] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.162719 3963 raft_consensus.cc:729] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.163010 3963 consensus_queue.cc:260] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.163666 3963 ts_tablet_manager.cc:1398] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.015s user 0.000s sys 0.002s
I20240503 02:13:34.164032 3963 tablet_bootstrap.cc:492] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.165678 3963 tablet_bootstrap.cc:654] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.166527 4109 raft_consensus.cc:482] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.166663 4109 raft_consensus.cc:504] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.167333 4109 leader_election.cc:288] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.167876 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.168303 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.168483 3844 raft_consensus.cc:2454] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
W20240503 02:13:34.168319 3886 leader_election.cc:341] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.169091 3885 leader_election.cc:302] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.169656 4010 raft_consensus.cc:2781] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.169736 4010 raft_consensus.cc:482] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.169837 4010 raft_consensus.cc:3037] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.173283 4010 raft_consensus.cc:504] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.174224 3770 raft_consensus.cc:3037] T 15eeedfc81114776a8614759c7741de9 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.177124 4013 raft_consensus.cc:482] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.177187 4010 leader_election.cc:288] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.177249 4013 raft_consensus.cc:504] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.177642 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.177781 4013 leader_election.cc:288] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.177816 3844 raft_consensus.cc:3037] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.178423 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:34.178836 3886 leader_election.cc:341] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.180192 3962 tablet_bootstrap.cc:492] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7: No bootstrap required, opened a new log
I20240503 02:13:34.180318 3962 ts_tablet_manager.cc:1367] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent bootstrapping tablet: real 0.113s user 0.002s sys 0.003s
I20240503 02:13:34.180459 3964 tablet_bootstrap.cc:492] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.180590 3964 ts_tablet_manager.cc:1367] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.007s
I20240503 02:13:34.180883 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.181072 3962 raft_consensus.cc:348] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.181243 3962 raft_consensus.cc:374] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.181300 3962 raft_consensus.cc:729] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.181340 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a17e29c044fcaab0eb6e836eca141" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.180424 3844 raft_consensus.cc:2454] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.181955 4010 consensus_queue.cc:1035] T 15eeedfc81114776a8614759c7741de9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.182185 3739 leader_election.cc:302] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.182350 3885 leader_election.cc:302] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.181502 3962 consensus_queue.cc:260] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.182690 4013 raft_consensus.cc:2726] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.182749 4109 raft_consensus.cc:2781] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.182871 4109 raft_consensus.cc:686] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:34.183148 4109 consensus_queue.cc:237] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.183945 3962 ts_tablet_manager.cc:1398] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20240503 02:13:34.184700 4134 raft_consensus.cc:482] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.184824 4134 raft_consensus.cc:504] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.185348 4134 leader_election.cc:288] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.185925 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ae769e64a54f46b68f4699ec8ffd05" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
W20240503 02:13:34.186303 3814 leader_election.cc:341] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.186937 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ae769e64a54f46b68f4699ec8ffd05" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:34.187422 3812 leader_election.cc:341] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.187548 3812 leader_election.cc:302] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7; no voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.187969 4134 raft_consensus.cc:2726] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.181389 3964 raft_consensus.cc:348] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.188567 3963 tablet_bootstrap.cc:492] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.188633 3964 raft_consensus.cc:374] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.188695 3964 raft_consensus.cc:729] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.188724 3963 ts_tablet_manager.cc:1367] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.007s
I20240503 02:13:34.189121 3964 consensus_queue.cc:260] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.189472 3963 raft_consensus.cc:348] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.189680 3963 raft_consensus.cc:374] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.189745 3963 raft_consensus.cc:729] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.189922 3963 consensus_queue.cc:260] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.190078 4013 raft_consensus.cc:482] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.190176 4013 raft_consensus.cc:504] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.190367 3963 ts_tablet_manager.cc:1398] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:34.190608 3963 tablet_bootstrap.cc:492] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.190692 4013 leader_election.cc:288] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.190956 4102 raft_consensus.cc:482] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.191072 4102 raft_consensus.cc:504] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.191624 4102 leader_election.cc:288] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.192122 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.192456 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bc3ed40ce7448a98251d25e2a09798" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.192672 3920 raft_consensus.cc:2379] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:34.192742 3963 tablet_bootstrap.cc:654] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.193166 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a25012564547caaa8289902be2b42f" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:34.193707 3886 leader_election.cc:341] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.194048 3739 leader_election.cc:302] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.194489 4013 raft_consensus.cc:3037] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.197840 4102 raft_consensus.cc:482] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.198004 4102 raft_consensus.cc:504] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.198760 4102 leader_election.cc:288] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.199337 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8fcb19cccf64822aa8ba1dd55bfc5a5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
W20240503 02:13:34.199805 3886 leader_election.cc:341] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.200476 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a25012564547caaa8289902be2b42f" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.200632 3844 raft_consensus.cc:2454] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.200950 3885 leader_election.cc:302] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.201027 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8fcb19cccf64822aa8ba1dd55bfc5a5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.201210 3846 raft_consensus.cc:2454] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.201396 4105 raft_consensus.cc:2781] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.201475 4105 raft_consensus.cc:482] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.201560 4105 raft_consensus.cc:3037] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.201994 4013 raft_consensus.cc:2726] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.203477 3964 ts_tablet_manager.cc:1398] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20240503 02:13:34.203758 3964 tablet_bootstrap.cc:492] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.204039 4105 raft_consensus.cc:504] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.204888 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a25012564547caaa8289902be2b42f" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:34.205312 3886 leader_election.cc:341] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.205387 3964 tablet_bootstrap.cc:654] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.205260 3671 catalog_manager.cc:5484] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.206207 3885 leader_election.cc:302] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.206766 4010 raft_consensus.cc:2781] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.206842 4010 raft_consensus.cc:482] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.206897 4010 raft_consensus.cc:3037] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.206915 4105 leader_election.cc:288] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.207324 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a25012564547caaa8289902be2b42f" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.207450 3846 raft_consensus.cc:3037] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.209280 4010 raft_consensus.cc:504] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.209841 3846 raft_consensus.cc:2454] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.209918 4010 leader_election.cc:288] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.206768 4109 consensus_queue.cc:1035] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20240503 02:13:34.210333 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8fcb19cccf64822aa8ba1dd55bfc5a5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
W20240503 02:13:34.210718 3886 leader_election.cc:341] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.210899 3963 tablet_bootstrap.cc:492] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.211040 3963 ts_tablet_manager.cc:1367] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.003s
I20240503 02:13:34.211251 3885 leader_election.cc:302] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.211877 3963 raft_consensus.cc:348] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.212029 3963 raft_consensus.cc:374] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.212081 3963 raft_consensus.cc:729] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.212231 3963 consensus_queue.cc:260] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.212687 4109 raft_consensus.cc:2781] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.212910 4109 raft_consensus.cc:686] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:34.213129 4109 consensus_queue.cc:237] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.213583 3963 ts_tablet_manager.cc:1398] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:34.213848 3963 tablet_bootstrap.cc:492] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.214854 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8fcb19cccf64822aa8ba1dd55bfc5a5" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.215049 3846 raft_consensus.cc:3037] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.215438 3963 tablet_bootstrap.cc:654] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
W20240503 02:13:34.221733 3886 consensus_peers.cc:485] T ddd36fffa3034c6cbe511f4f39b078ff P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Couldn't send request to peer 6e0d1128f0404d609df72fd607c20650. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:34.224557 3671 catalog_manager.cc:5484] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.230640 3964 tablet_bootstrap.cc:492] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.230772 3964 ts_tablet_manager.cc:1367] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.007s
I20240503 02:13:34.231433 3964 raft_consensus.cc:348] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.231621 3964 raft_consensus.cc:374] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.231679 3964 raft_consensus.cc:729] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.231757 3846 raft_consensus.cc:2454] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.231861 3964 consensus_queue.cc:260] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.232146 3885 leader_election.cc:302] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.232414 3964 ts_tablet_manager.cc:1398] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:34.232472 4010 raft_consensus.cc:2781] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.232597 4010 raft_consensus.cc:686] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:34.232775 3964 tablet_bootstrap.cc:492] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.232846 4010 consensus_queue.cc:237] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.234465 3964 tablet_bootstrap.cc:654] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.242230 3963 tablet_bootstrap.cc:492] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.242349 3963 ts_tablet_manager.cc:1367] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.000s
I20240503 02:13:34.242856 3963 raft_consensus.cc:348] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.242981 3963 raft_consensus.cc:374] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.243031 3963 raft_consensus.cc:729] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.243208 3963 consensus_queue.cc:260] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.243662 3963 ts_tablet_manager.cc:1398] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:13:34.243870 3963 tablet_bootstrap.cc:492] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.243969 3964 tablet_bootstrap.cc:492] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.244100 3668 catalog_manager.cc:5484] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.245383 3963 tablet_bootstrap.cc:654] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.245458 4134 consensus_queue.cc:1035] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.244097 3964 ts_tablet_manager.cc:1367] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20240503 02:13:34.253401 3964 raft_consensus.cc:348] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.253630 3964 raft_consensus.cc:374] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.253686 3964 raft_consensus.cc:729] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.255044 3964 consensus_queue.cc:260] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.255753 3964 ts_tablet_manager.cc:1398] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:13:34.256049 3964 tablet_bootstrap.cc:492] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.257876 3964 tablet_bootstrap.cc:654] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.263665 3963 tablet_bootstrap.cc:492] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.263787 3963 ts_tablet_manager.cc:1367] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.012s
I20240503 02:13:34.264036 3984 consensus_queue.cc:1035] T d9da09c39261443bbf898a390937ffb4 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.264501 3963 raft_consensus.cc:348] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.264681 3963 raft_consensus.cc:374] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.264743 3963 raft_consensus.cc:729] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.264953 3963 consensus_queue.cc:260] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.270181 3964 tablet_bootstrap.cc:492] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.270133 4010 consensus_queue.cc:1035] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.270294 3964 ts_tablet_manager.cc:1367] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20240503 02:13:34.272931 3964 raft_consensus.cc:348] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.273139 3964 raft_consensus.cc:374] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.273221 3964 raft_consensus.cc:729] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.273393 3964 consensus_queue.cc:260] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.273856 3963 ts_tablet_manager.cc:1398] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20240503 02:13:34.273947 3964 ts_tablet_manager.cc:1398] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240503 02:13:34.274266 3964 tablet_bootstrap.cc:492] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.275736 3963 tablet_bootstrap.cc:492] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.275897 3964 tablet_bootstrap.cc:654] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.277875 3963 tablet_bootstrap.cc:654] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.282076 4010 consensus_queue.cc:1035] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.285288 3770 raft_consensus.cc:3037] T b04a17e29c044fcaab0eb6e836eca141 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.321698 3963 tablet_bootstrap.cc:492] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.321873 3963 ts_tablet_manager.cc:1367] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.046s user 0.017s sys 0.012s
I20240503 02:13:34.322633 3963 raft_consensus.cc:348] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.323004 3963 raft_consensus.cc:374] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.323158 3963 raft_consensus.cc:729] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.323395 3963 consensus_queue.cc:260] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.324301 3963 ts_tablet_manager.cc:1398] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:34.325575 3963 tablet_bootstrap.cc:492] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe: Bootstrap starting.
I20240503 02:13:34.325704 3984 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.334767 4105 raft_consensus.cc:482] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.335180 4105 raft_consensus.cc:504] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.336066 4105 leader_election.cc:288] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.336294 3963 tablet_bootstrap.cc:654] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.336436 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df08ada758d5442cba4f7bd85b445352" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.336727 3964 tablet_bootstrap.cc:492] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.336776 3846 raft_consensus.cc:2454] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.336886 3964 ts_tablet_manager.cc:1367] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.063s user 0.023s sys 0.010s
I20240503 02:13:34.337235 3885 leader_election.cc:302] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.337544 4105 raft_consensus.cc:2781] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.337661 4105 raft_consensus.cc:482] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.337790 4105 raft_consensus.cc:3037] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.337786 3964 raft_consensus.cc:348] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.338029 3964 raft_consensus.cc:374] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.337863 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df08ada758d5442cba4f7bd85b445352" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.338162 3964 raft_consensus.cc:729] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.337983 4134 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.338467 3771 raft_consensus.cc:2454] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.340082 4105 raft_consensus.cc:504] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.340947 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df08ada758d5442cba4f7bd85b445352" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.340946 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df08ada758d5442cba4f7bd85b445352" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:34.341423 3964 consensus_queue.cc:260] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.343329 3846 raft_consensus.cc:3037] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.348325 4134 raft_consensus.cc:482] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.348358 4105 leader_election.cc:288] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.348830 3846 raft_consensus.cc:2454] T df08ada758d5442cba4f7bd85b445352 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.349244 4134 raft_consensus.cc:504] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.349684 3964 ts_tablet_manager.cc:1398] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240503 02:13:34.350643 3885 leader_election.cc:302] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.351356 3771 raft_consensus.cc:3037] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.352035 3964 tablet_bootstrap.cc:492] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.353461 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8c6d907cff43638bf43b45e1c3edd0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.353593 4134 leader_election.cc:288] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.354097 4105 raft_consensus.cc:2781] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.355628 3771 raft_consensus.cc:2454] T df08ada758d5442cba4f7bd85b445352 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.357648 4105 raft_consensus.cc:686] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
W20240503 02:13:34.357712 3812 leader_election.cc:341] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.358850 4105 consensus_queue.cc:237] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.358500 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8c6d907cff43638bf43b45e1c3edd0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.359383 3963 tablet_bootstrap.cc:492] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe: No bootstrap required, opened a new log
I20240503 02:13:34.360149 3963 ts_tablet_manager.cc:1367] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.010s
I20240503 02:13:34.361627 3920 raft_consensus.cc:2454] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 0.
I20240503 02:13:34.362080 3814 leader_election.cc:302] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters: 6e0d1128f0404d609df72fd607c20650
I20240503 02:13:34.362440 4134 raft_consensus.cc:2781] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.363034 4134 raft_consensus.cc:482] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.363099 3963 raft_consensus.cc:348] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.363197 4134 raft_consensus.cc:3037] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.363607 3964 tablet_bootstrap.cc:654] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.366412 4134 raft_consensus.cc:504] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.367300 4134 leader_election.cc:288] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.367694 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8c6d907cff43638bf43b45e1c3edd0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.367900 3920 raft_consensus.cc:3037] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.368268 3963 raft_consensus.cc:374] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.368362 3963 raft_consensus.cc:729] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.368597 3963 consensus_queue.cc:260] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.369504 3963 ts_tablet_manager.cc:1398] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20240503 02:13:34.370342 3920 raft_consensus.cc:2454] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:34.370747 3814 leader_election.cc:302] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.370815 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8c6d907cff43638bf43b45e1c3edd0" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:34.371080 4134 raft_consensus.cc:2781] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election won for term 1
W20240503 02:13:34.371357 3812 leader_election.cc:341] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:13:34.371620 4134 raft_consensus.cc:686] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:34.372176 3668 catalog_manager.cc:5484] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.373034 3964 tablet_bootstrap.cc:492] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.373239 3964 ts_tablet_manager.cc:1367] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.000s
I20240503 02:13:34.373034 4134 consensus_queue.cc:237] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.373965 3964 raft_consensus.cc:348] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.374238 3964 raft_consensus.cc:374] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.374352 3964 raft_consensus.cc:729] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.374653 3964 consensus_queue.cc:260] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.375299 3964 ts_tablet_manager.cc:1398] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:34.375756 3964 tablet_bootstrap.cc:492] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.378217 3964 tablet_bootstrap.cc:654] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.383791 3668 catalog_manager.cc:5484] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.387815 3771 raft_consensus.cc:3037] T 2758e6b755a9470e872f11fe2fb9001a P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.390568 4109 consensus_queue.cc:1035] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.392858 3964 tablet_bootstrap.cc:492] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.394313 3964 ts_tablet_manager.cc:1367] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.002s
I20240503 02:13:34.395005 3964 raft_consensus.cc:348] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.395176 3964 raft_consensus.cc:374] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.395303 3964 raft_consensus.cc:729] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.395520 3964 consensus_queue.cc:260] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.395912 4102 consensus_queue.cc:1035] T 2758e6b755a9470e872f11fe2fb9001a P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.396096 3964 ts_tablet_manager.cc:1398] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240503 02:13:34.396373 3964 tablet_bootstrap.cc:492] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.397681 3964 tablet_bootstrap.cc:654] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.399376 3964 tablet_bootstrap.cc:492] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.399559 3964 ts_tablet_manager.cc:1367] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240503 02:13:34.400182 3964 raft_consensus.cc:348] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.400362 3964 raft_consensus.cc:374] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.400451 3964 raft_consensus.cc:729] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.400681 3964 consensus_queue.cc:260] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.401211 3964 ts_tablet_manager.cc:1398] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240503 02:13:34.401473 3964 tablet_bootstrap.cc:492] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.403084 3964 tablet_bootstrap.cc:654] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.404651 3964 tablet_bootstrap.cc:492] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.404799 3964 ts_tablet_manager.cc:1367] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240503 02:13:34.405333 3964 raft_consensus.cc:348] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.405517 3964 raft_consensus.cc:374] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.405656 3964 raft_consensus.cc:729] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.405869 3964 consensus_queue.cc:260] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.406383 3964 ts_tablet_manager.cc:1398] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240503 02:13:34.406654 3964 tablet_bootstrap.cc:492] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.408196 3964 tablet_bootstrap.cc:654] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.409451 3771 raft_consensus.cc:3037] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.414135 4102 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.420322 3964 tablet_bootstrap.cc:492] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.420480 3964 ts_tablet_manager.cc:1367] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20240503 02:13:34.421249 3964 raft_consensus.cc:348] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.421480 3964 raft_consensus.cc:374] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.421607 3964 raft_consensus.cc:729] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.421815 3964 consensus_queue.cc:260] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.422220 3964 ts_tablet_manager.cc:1398] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:34.422444 3964 tablet_bootstrap.cc:492] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650: Bootstrap starting.
I20240503 02:13:34.423210 4266 raft_consensus.cc:482] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.423352 4266 raft_consensus.cc:504] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.423538 4109 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.423890 3964 tablet_bootstrap.cc:654] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650: Neither blocks nor log segments found. Creating new log.
I20240503 02:13:34.424000 3771 raft_consensus.cc:3037] T 793fe86859754f3d90919e150e2f3693 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.424041 4266 leader_election.cc:288] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.424767 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a511b3bba2024783a999d7c7b616100e" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.425024 3846 raft_consensus.cc:2454] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:34.425693 3736 leader_election.cc:302] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.426290 4013 raft_consensus.cc:2781] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.426330 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a511b3bba2024783a999d7c7b616100e" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.426437 4013 raft_consensus.cc:482] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.426543 3920 raft_consensus.cc:2454] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:34.426715 4016 raft_consensus.cc:482] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.426888 4016 raft_consensus.cc:504] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.427662 4016 leader_election.cc:288] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.426549 4013 raft_consensus.cc:3037] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.433655 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a25012564547caaa8289902be2b42f" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.434196 3842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86a25012564547caaa8289902be2b42f" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.434523 3842 raft_consensus.cc:2379] T 86a25012564547caaa8289902be2b42f P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:34.434429 4134 consensus_queue.cc:1035] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.435191 3975 consensus_queue.cc:1035] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.435647 4016 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.435808 4016 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.436376 4016 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.436802 3736 leader_election.cc:302] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.436870 4013 raft_consensus.cc:504] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.437134 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.437333 3921 raft_consensus.cc:2454] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 2.
I20240503 02:13:34.437503 4013 leader_election.cc:288] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.437705 4013 raft_consensus.cc:3037] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.438519 3739 leader_election.cc:302] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.438946 3975 raft_consensus.cc:2781] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240503 02:13:34.439036 3975 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.439129 3975 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:34.439541 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a511b3bba2024783a999d7c7b616100e" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.439761 3841 raft_consensus.cc:3037] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.440937 4013 raft_consensus.cc:2726] T 86a25012564547caaa8289902be2b42f P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.441800 4266 consensus_queue.cc:1035] T f6ff048917b34a4b9cd09dee267fc078 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.438531 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.442533 3841 raft_consensus.cc:2454] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.442814 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a511b3bba2024783a999d7c7b616100e" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.443022 3920 raft_consensus.cc:3037] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.443622 4134 consensus_queue.cc:1035] T ea39b9e4e11247ab95cc690e60b22b11 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.444018 3736 leader_election.cc:302] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.444561 4266 raft_consensus.cc:2781] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.444720 4266 raft_consensus.cc:686] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:34.445036 4266 consensus_queue.cc:237] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.445991 3975 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.446895 3975 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 3 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.447568 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.448130 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.448745 3922 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:13:34.448516 3920 raft_consensus.cc:2454] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.450467 4016 raft_consensus.cc:482] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.450742 4016 raft_consensus.cc:504] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.451447 4016 leader_election.cc:288] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.452020 3922 raft_consensus.cc:2454] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 3.
I20240503 02:13:34.452308 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.453114 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9aab321bd48c38b354316119370d2" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.453315 3739 leader_election.cc:302] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4001ccb96b064ad2a2ca9c61096516fe, 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7
I20240503 02:13:34.453991 3975 raft_consensus.cc:2781] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:13:34.454245 3975 raft_consensus.cc:686] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 3 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:34.454680 3975 consensus_queue.cc:237] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.455375 3739 leader_election.cc:302] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.456838 4016 raft_consensus.cc:2726] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.460748 3964 tablet_bootstrap.cc:492] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650: No bootstrap required, opened a new log
I20240503 02:13:34.460958 3964 ts_tablet_manager.cc:1367] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.007s
I20240503 02:13:34.461841 3964 raft_consensus.cc:348] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.462116 3964 raft_consensus.cc:374] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:13:34.462273 3964 raft_consensus.cc:729] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Initialized, Role: FOLLOWER
I20240503 02:13:34.462553 3964 consensus_queue.cc:260] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.463297 3964 ts_tablet_manager.cc:1398] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:34.472865 3975 raft_consensus.cc:482] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.473024 3975 raft_consensus.cc:504] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.474185 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd36fffa3034c6cbe511f4f39b078ff" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.474942 3975 leader_election.cc:288] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.475538 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd36fffa3034c6cbe511f4f39b078ff" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.476049 4266 raft_consensus.cc:482] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.476219 4266 raft_consensus.cc:504] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.476344 3739 leader_election.cc:302] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.476687 4013 raft_consensus.cc:2726] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.476810 3669 catalog_manager.cc:5484] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.477335 3669 catalog_manager.cc:5484] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 1 to 3, leader changed from 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 3 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.478138 4266 leader_election.cc:288] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.478628 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4972943b854b28a0c927ce1a0511d7" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.478816 3844 raft_consensus.cc:2454] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:34.478935 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4972943b854b28a0c927ce1a0511d7" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.479108 3919 raft_consensus.cc:2454] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 0.
I20240503 02:13:34.479699 3736 leader_election.cc:302] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.480532 4266 raft_consensus.cc:2781] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.480612 4266 raft_consensus.cc:482] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.480671 4266 raft_consensus.cc:3037] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.484259 4266 raft_consensus.cc:504] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.485248 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4972943b854b28a0c927ce1a0511d7" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.485489 3844 raft_consensus.cc:3037] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.488899 3844 raft_consensus.cc:2454] T 7f4972943b854b28a0c927ce1a0511d7 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.489295 3736 leader_election.cc:302] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.489570 3975 raft_consensus.cc:2781] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.489711 4266 leader_election.cc:288] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 election: Requested vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.489836 3975 raft_consensus.cc:686] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0d1128f0404d609df72fd607c20650, State: Running, Role: LEADER
I20240503 02:13:34.490095 3975 consensus_queue.cc:237] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.490535 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4972943b854b28a0c927ce1a0511d7" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.490685 3919 raft_consensus.cc:3037] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.492626 3919 raft_consensus.cc:2454] T 7f4972943b854b28a0c927ce1a0511d7 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e0d1128f0404d609df72fd607c20650 in term 1.
I20240503 02:13:34.495301 3671 catalog_manager.cc:5484] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e0d1128f0404d609df72fd607c20650 (127.2.157.65). New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.522354 3769 raft_consensus.cc:3037] T eef59225fda142b99782f8e09d65cb9e P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.524791 3967 consensus_queue.cc:1035] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.529865 4094 consensus_queue.cc:1035] T eef59225fda142b99782f8e09d65cb9e P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.540740 3769 raft_consensus.cc:3037] T a5e9aab321bd48c38b354316119370d2 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.556973 4094 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.557176 4094 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.557931 4094 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.558176 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.558418 3769 raft_consensus.cc:2454] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:34.558459 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.558606 3919 raft_consensus.cc:2454] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 1.
I20240503 02:13:34.558835 3812 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 6e0d1128f0404d609df72fd607c20650; no voters:
I20240503 02:13:34.559092 4094 raft_consensus.cc:2781] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240503 02:13:34.559190 4094 raft_consensus.cc:482] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.559273 4094 raft_consensus.cc:3037] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:34.562650 4094 raft_consensus.cc:504] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.563537 4094 leader_election.cc:288] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.563640 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:34.563750 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dc34edc2e42aab621736c3ae036e3" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe"
I20240503 02:13:34.563802 3769 raft_consensus.cc:3037] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:34.563948 3919 raft_consensus.cc:3037] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:13:34.566438 3919 raft_consensus.cc:2454] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 2.
I20240503 02:13:34.566438 3769 raft_consensus.cc:2454] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in term 2.
I20240503 02:13:34.566886 3814 leader_election.cc:302] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.567224 4094 raft_consensus.cc:2781] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:13:34.567320 4094 raft_consensus.cc:686] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 2 LEADER]: Becoming Leader. State: Replica: 2c8f35a9c7534ec2b0251e826a6a9da7, State: Running, Role: LEADER
I20240503 02:13:34.567543 4094 consensus_queue.cc:237] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.570466 3975 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.571472 3669 catalog_manager.cc:5484] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: term changed from 0 to 2, leader changed from <none> to 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66). New cstate: current_term: 2 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:34.572739 3769 raft_consensus.cc:3037] T 4bc392ed1bce416c8e0d47af8c9787bf P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.575639 4109 consensus_queue.cc:1035] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.577154 3975 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.580855 4109 raft_consensus.cc:482] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.580988 4109 raft_consensus.cc:504] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.581661 4109 leader_election.cc:288] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.582219 3844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ae769e64a54f46b68f4699ec8ffd05" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.582437 3844 raft_consensus.cc:2454] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.582933 3885 leader_election.cc:302] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.583555 4235 raft_consensus.cc:2781] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:13:34.583657 4235 raft_consensus.cc:482] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:13:34.583745 4235 raft_consensus.cc:3037] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.587045 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ae769e64a54f46b68f4699ec8ffd05" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.587283 3771 raft_consensus.cc:2454] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 0.
I20240503 02:13:34.590443 4010 consensus_queue.cc:1035] T 4bc392ed1bce416c8e0d47af8c9787bf P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.594125 4235 raft_consensus.cc:504] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.595229 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ae769e64a54f46b68f4699ec8ffd05" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7"
I20240503 02:13:34.595422 3846 raft_consensus.cc:3037] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.596921 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ae769e64a54f46b68f4699ec8ffd05" candidate_uuid: "4001ccb96b064ad2a2ca9c61096516fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650"
I20240503 02:13:34.597110 3769 raft_consensus.cc:3037] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.599153 3846 raft_consensus.cc:2454] T 05ae769e64a54f46b68f4699ec8ffd05 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.600328 3885 leader_election.cc:302] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe; no voters:
I20240503 02:13:34.600782 4102 raft_consensus.cc:2781] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:13:34.600951 4102 raft_consensus.cc:686] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Becoming Leader. State: Replica: 4001ccb96b064ad2a2ca9c61096516fe, State: Running, Role: LEADER
I20240503 02:13:34.601217 4102 consensus_queue.cc:237] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.601325 3769 raft_consensus.cc:2454] T 05ae769e64a54f46b68f4699ec8ffd05 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4001ccb96b064ad2a2ca9c61096516fe in term 1.
I20240503 02:13:34.603826 4235 leader_election.cc:288] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [CANDIDATE]: Term 1 election: Requested vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:34.605000 4094 raft_consensus.cc:482] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4001ccb96b064ad2a2ca9c61096516fe)
I20240503 02:13:34.605152 4094 raft_consensus.cc:504] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.605899 4094 leader_election.cc:288] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.606256 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.606493 3921 raft_consensus.cc:2354] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 for earlier term 2. Current term is 3.
I20240503 02:13:34.606444 3769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43c2f4090f14b4baaf2a55a857ed245" candidate_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e0d1128f0404d609df72fd607c20650" is_pre_election: true
I20240503 02:13:34.606928 3814 leader_election.cc:398] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983) with higher term. Message: Invalid argument: T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 for earlier term 2. Current term is 3.
I20240503 02:13:34.606994 3814 leader_election.cc:401] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20240503 02:13:34.607168 4094 raft_consensus.cc:3037] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Advancing to term 3
I20240503 02:13:34.610409 4094 raft_consensus.cc:2726] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983) with higher term. Message: Invalid argument: T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c8f35a9c7534ec2b0251e826a6a9da7 for earlier term 2. Current term is 3.
I20240503 02:13:34.611478 3669 catalog_manager.cc:5484] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67). New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } }
I20240503 02:13:34.635726 3769 raft_consensus.cc:3037] T ddd36fffa3034c6cbe511f4f39b078ff P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.648447 4013 raft_consensus.cc:482] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.648653 4013 raft_consensus.cc:504] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.649590 4013 leader_election.cc:288] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.649834 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.650059 3841 raft_consensus.cc:2379] T 942fa19a821249e1b11d625a6f712368 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 4001ccb96b064ad2a2ca9c61096516fe in this term.
I20240503 02:13:34.650034 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "942fa19a821249e1b11d625a6f712368" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.650575 3739 leader_election.cc:302] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.650818 4013 raft_consensus.cc:3037] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.654055 4013 raft_consensus.cc:2726] T 942fa19a821249e1b11d625a6f712368 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.655472 4235 consensus_queue.cc:1035] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.660377 4102 consensus_queue.cc:1035] T 942fa19a821249e1b11d625a6f712368 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.689499 4105 consensus_queue.cc:1035] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.693580 4010 consensus_queue.cc:1035] T 86a25012564547caaa8289902be2b42f P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.760242 4102 consensus_queue.cc:1035] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.764829 3769 raft_consensus.cc:3037] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.768891 4109 consensus_queue.cc:1035] T e8fcb19cccf64822aa8ba1dd55bfc5a5 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.777660 4013 raft_consensus.cc:482] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:13:34.777778 4013 raft_consensus.cc:504] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } }
I20240503 02:13:34.778276 4013 leader_election.cc:288] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979), 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:34.778630 3841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8c6d907cff43638bf43b45e1c3edd0" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" is_pre_election: true
I20240503 02:13:34.778630 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8c6d907cff43638bf43b45e1c3edd0" candidate_uuid: "6e0d1128f0404d609df72fd607c20650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4001ccb96b064ad2a2ca9c61096516fe" is_pre_election: true
I20240503 02:13:34.778859 3919 raft_consensus.cc:2379] T 0b8c6d907cff43638bf43b45e1c3edd0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e0d1128f0404d609df72fd607c20650 in current term 1: Already voted for candidate 2c8f35a9c7534ec2b0251e826a6a9da7 in this term.
I20240503 02:13:34.779229 3739 leader_election.cc:302] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e0d1128f0404d609df72fd607c20650; no voters: 2c8f35a9c7534ec2b0251e826a6a9da7, 4001ccb96b064ad2a2ca9c61096516fe
I20240503 02:13:34.779489 4013 raft_consensus.cc:3037] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:13:34.781419 4013 raft_consensus.cc:2726] T 0b8c6d907cff43638bf43b45e1c3edd0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:13:34.786717 4109 consensus_queue.cc:1035] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.790203 4105 consensus_queue.cc:1035] T df08ada758d5442cba4f7bd85b445352 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.815735 3967 consensus_queue.cc:1035] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.820288 3984 consensus_queue.cc:1035] T 0b8c6d907cff43638bf43b45e1c3edd0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.855201 3844 raft_consensus.cc:1261] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 3 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20240503 02:13:34.855857 3975 consensus_queue.cc:1035] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:34.859575 3919 raft_consensus.cc:1261] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20240503 02:13:34.860067 4266 consensus_queue.cc:1035] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:34.952237 4266 consensus_queue.cc:1035] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.955207 4013 consensus_queue.cc:1035] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.957933 3975 consensus_queue.cc:1035] T 7f4972943b854b28a0c927ce1a0511d7 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.968642 4266 consensus_queue.cc:1035] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:34.993121 3921 raft_consensus.cc:1261] T ba1dc34edc2e42aab621736c3ae036e3 P 4001ccb96b064ad2a2ca9c61096516fe [term 2 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240503 02:13:34.993667 4066 consensus_queue.cc:1035] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:35.001032 3769 raft_consensus.cc:1261] T ba1dc34edc2e42aab621736c3ae036e3 P 6e0d1128f0404d609df72fd607c20650 [term 2 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240503 02:13:35.001560 4066 consensus_queue.cc:1035] T ba1dc34edc2e42aab621736c3ae036e3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:35.106108 4010 consensus_queue.cc:1035] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:35.109633 4109 consensus_queue.cc:1035] T 05ae769e64a54f46b68f4699ec8ffd05 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:13:35.335294 2677 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:13:35.338541 4366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:35.355475 4369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:13:35.356169 4367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:13:35.356756 2677 server_base.cc:1034] running on GCE node
I20240503 02:13:35.357055 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240503 02:13:35.357111 2677 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240503 02:13:35.357147 2677 hybrid_clock.cc:648] HybridClock initialized: now 1714702415357144 us; error 0 us; skew 500 ppm
I20240503 02:13:35.357267 2677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:13:35.358590 2677 webserver.cc:460] Webserver started at http://127.2.157.68:45065/ using document root <none> and password file <none>
I20240503 02:13:35.358808 2677 fs_manager.cc:359] Metadata directory not provided
I20240503 02:13:35.358865 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:13:35.358968 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:13:35.359588 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-3-root/instance:
uuid: "5be54f33baf0422ab4ebf2f11e01e108"
format_stamp: "Formatted at 2024-05-03 02:13:35 on dist-test-slave-tz5s"
I20240503 02:13:35.362185 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:35.364547 4374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:35.365388 2677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:35.365550 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-3-root
uuid: "5be54f33baf0422ab4ebf2f11e01e108"
format_stamp: "Formatted at 2024-05-03 02:13:35 on dist-test-slave-tz5s"
I20240503 02:13:35.365674 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:13:35.387807 2677 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:13:35.388399 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:13:35.389079 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:13:35.390111 2677 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:13:35.390169 2677 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:35.390254 2677 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:13:35.390304 2677 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:13:35.403553 2677 rpc_server.cc:307] RPC server started. Bound to: 127.2.157.68:41241
I20240503 02:13:35.403566 4436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.157.68:41241 every 8 connection(s)
I20240503 02:13:35.404575 2677 auto_leader_rebalancer-test.cc:405] add a tserver: 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:35.409756 4437 heartbeater.cc:347] Connected to a master server at 127.2.157.126:40759
I20240503 02:13:35.409883 4437 heartbeater.cc:464] Registering TS with master...
I20240503 02:13:35.410179 4437 heartbeater.cc:511] Master 127.2.157.126:40759 requested a full tablet report, sending...
I20240503 02:13:35.410907 3668 ts_manager.cc:194] Registered new tserver with Master: 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241)
I20240503 02:13:35.411651 3668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34662
I20240503 02:13:36.014039 3841 consensus_queue.cc:237] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.016113 3769 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.016398 3919 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.016531 3967 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.016964 3984 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.030176 4066 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.030728 3919 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.033516 3769 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.036118 3841 consensus_queue.cc:237] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
W20240503 02:13:36.039755 3811 consensus_peers.cc:485] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34af2cfd01c7406da5ed8da1ce0cf129. This is attempt 1: this message will repeat every 5th retry.
W20240503 02:13:36.040081 3811 consensus_peers.cc:485] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c1aea07837f41a281d484e37eab5744. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:36.040625 3668 catalog_manager.cc:5484] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.041282 3919 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.041775 3967 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.042500 3769 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.043053 3967 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.044840 3984 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.045676 3919 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.057358 3769 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.067868 3919 consensus_queue.cc:237] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.070617 3769 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.071496 3668 catalog_manager.cc:5484] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.077064 3841 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.077523 4109 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.077833 4010 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.084408 4105 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.084861 3841 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.089340 3668 catalog_manager.cc:5484] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
W20240503 02:13:36.090507 3885 consensus_peers.cc:485] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 518cefb57371493496e276f5a10fd53d. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:36.090780 3844 consensus_queue.cc:237] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.093291 3769 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
W20240503 02:13:36.097712 3811 consensus_peers.cc:485] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1101bf1e3c82448ebf8747b8533fd2f0. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:36.098305 3919 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.098485 3769 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:36.098935 4066 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.099560 3967 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:36.101725 3967 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.102110 3921 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.106891 3668 catalog_manager.cc:5484] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.114181 3771 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:36.412619 4437 heartbeater.cc:503] Master 127.2.157.126:40759 was elected leader, sending a full tablet report...
I20240503 02:13:36.487424 4465 ts_tablet_manager.cc:902] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:36.489576 4465 tablet_copy_client.cc:308] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.66:43979
I20240503 02:13:36.498399 3858 tablet_copy_service.cc:140] P 2c8f35a9c7534ec2b0251e826a6a9da7: Received BeginTabletCopySession request for tablet 1101bf1e3c82448ebf8747b8533fd2f0 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:43200)
I20240503 02:13:36.498622 3858 tablet_copy_service.cc:161] P 2c8f35a9c7534ec2b0251e826a6a9da7: Beginning new tablet copy session on tablet 1101bf1e3c82448ebf8747b8533fd2f0 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:43200: session id = 5be54f33baf0422ab4ebf2f11e01e108-1101bf1e3c82448ebf8747b8533fd2f0
I20240503 02:13:36.501946 3858 tablet_copy_source_session.cc:205] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:36.503686 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1101bf1e3c82448ebf8747b8533fd2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:36.509908 4465 tablet_copy_client.cc:783] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:36.510234 4465 tablet_copy_client.cc:647] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:36.512598 4465 tablet_copy_client.cc:523] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:36.516520 4465 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:36.523434 4465 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:36.523813 4465 tablet_bootstrap.cc:492] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:36.524037 4465 ts_tablet_manager.cc:1367] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20240503 02:13:36.524739 4465 raft_consensus.cc:348] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.524914 4465 raft_consensus.cc:729] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:36.525091 4465 consensus_queue.cc:260] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.525914 4465 ts_tablet_manager.cc:1398] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20240503 02:13:36.526609 3858 tablet_copy_service.cc:342] P 2c8f35a9c7534ec2b0251e826a6a9da7: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-1101bf1e3c82448ebf8747b8533fd2f0 received from {username='slave'} at 127.0.0.1:43200
I20240503 02:13:36.526727 3858 tablet_copy_service.cc:433] P 2c8f35a9c7534ec2b0251e826a6a9da7: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-1101bf1e3c82448ebf8747b8533fd2f0 on tablet 1101bf1e3c82448ebf8747b8533fd2f0 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:36.561069 4465 ts_tablet_manager.cc:902] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:36.562377 4465 tablet_copy_client.cc:308] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.67:34983
I20240503 02:13:36.568305 3932 tablet_copy_service.cc:140] P 4001ccb96b064ad2a2ca9c61096516fe: Received BeginTabletCopySession request for tablet 518cefb57371493496e276f5a10fd53d from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:46886)
I20240503 02:13:36.568457 3932 tablet_copy_service.cc:161] P 4001ccb96b064ad2a2ca9c61096516fe: Beginning new tablet copy session on tablet 518cefb57371493496e276f5a10fd53d from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:46886: session id = 5be54f33baf0422ab4ebf2f11e01e108-518cefb57371493496e276f5a10fd53d
I20240503 02:13:36.571092 3932 tablet_copy_source_session.cc:205] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:36.572229 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518cefb57371493496e276f5a10fd53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:36.577901 4465 tablet_copy_client.cc:783] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:36.578117 4465 tablet_copy_client.cc:647] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:36.579564 4465 tablet_copy_client.cc:523] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:36.582424 4465 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:36.587399 4465 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:36.587806 4465 tablet_bootstrap.cc:492] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:36.588014 4465 ts_tablet_manager.cc:1367] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20240503 02:13:36.588668 4465 raft_consensus.cc:348] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.588915 4465 raft_consensus.cc:729] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:36.589141 4465 consensus_queue.cc:260] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.589744 4465 ts_tablet_manager.cc:1398] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240503 02:13:36.590590 3932 tablet_copy_service.cc:342] P 4001ccb96b064ad2a2ca9c61096516fe: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-518cefb57371493496e276f5a10fd53d received from {username='slave'} at 127.0.0.1:46886
I20240503 02:13:36.590688 3932 tablet_copy_service.cc:433] P 4001ccb96b064ad2a2ca9c61096516fe: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-518cefb57371493496e276f5a10fd53d on tablet 518cefb57371493496e276f5a10fd53d with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:36.604732 4465 ts_tablet_manager.cc:902] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:36.606160 4465 tablet_copy_client.cc:308] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.66:43979
I20240503 02:13:36.606781 3858 tablet_copy_service.cc:140] P 2c8f35a9c7534ec2b0251e826a6a9da7: Received BeginTabletCopySession request for tablet 8c1aea07837f41a281d484e37eab5744 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:43200)
I20240503 02:13:36.606915 3858 tablet_copy_service.cc:161] P 2c8f35a9c7534ec2b0251e826a6a9da7: Beginning new tablet copy session on tablet 8c1aea07837f41a281d484e37eab5744 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:43200: session id = 5be54f33baf0422ab4ebf2f11e01e108-8c1aea07837f41a281d484e37eab5744
I20240503 02:13:36.609658 3858 tablet_copy_source_session.cc:205] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:36.611006 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1aea07837f41a281d484e37eab5744. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:36.616194 4465 tablet_copy_client.cc:783] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:36.616390 4465 tablet_copy_client.cc:647] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:36.618059 4465 tablet_copy_client.cc:523] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:36.621277 4465 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:36.626430 4465 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:36.626888 4465 tablet_bootstrap.cc:492] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:36.627116 4465 ts_tablet_manager.cc:1367] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.007s sys 0.000s
I20240503 02:13:36.627815 4465 raft_consensus.cc:348] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.628043 4465 raft_consensus.cc:729] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:36.628191 4465 consensus_queue.cc:260] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.628816 4465 ts_tablet_manager.cc:1398] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:36.629511 3858 tablet_copy_service.cc:342] P 2c8f35a9c7534ec2b0251e826a6a9da7: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-8c1aea07837f41a281d484e37eab5744 received from {username='slave'} at 127.0.0.1:43200
I20240503 02:13:36.629649 3858 tablet_copy_service.cc:433] P 2c8f35a9c7534ec2b0251e826a6a9da7: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-8c1aea07837f41a281d484e37eab5744 on tablet 8c1aea07837f41a281d484e37eab5744 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:36.653649 4465 ts_tablet_manager.cc:902] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:36.655750 4465 tablet_copy_client.cc:308] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.66:43979
I20240503 02:13:36.656301 3858 tablet_copy_service.cc:140] P 2c8f35a9c7534ec2b0251e826a6a9da7: Received BeginTabletCopySession request for tablet 34af2cfd01c7406da5ed8da1ce0cf129 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:43200)
I20240503 02:13:36.656438 3858 tablet_copy_service.cc:161] P 2c8f35a9c7534ec2b0251e826a6a9da7: Beginning new tablet copy session on tablet 34af2cfd01c7406da5ed8da1ce0cf129 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:43200: session id = 5be54f33baf0422ab4ebf2f11e01e108-34af2cfd01c7406da5ed8da1ce0cf129
I20240503 02:13:36.659279 3858 tablet_copy_source_session.cc:205] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:36.660687 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34af2cfd01c7406da5ed8da1ce0cf129. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:36.677047 4465 tablet_copy_client.cc:783] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:36.677309 4465 tablet_copy_client.cc:647] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:36.678647 4465 tablet_copy_client.cc:523] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:36.681402 4465 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:36.686482 4465 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:36.686828 4465 tablet_bootstrap.cc:492] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:36.687074 4465 ts_tablet_manager.cc:1367] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20240503 02:13:36.687721 4465 raft_consensus.cc:348] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.687888 4465 raft_consensus.cc:729] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:36.688030 4465 consensus_queue.cc:260] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:36.688566 4465 ts_tablet_manager.cc:1398] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:13:36.689507 3858 tablet_copy_service.cc:342] P 2c8f35a9c7534ec2b0251e826a6a9da7: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-34af2cfd01c7406da5ed8da1ce0cf129 received from {username='slave'} at 127.0.0.1:43200
I20240503 02:13:36.689640 3858 tablet_copy_service.cc:433] P 2c8f35a9c7534ec2b0251e826a6a9da7: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-34af2cfd01c7406da5ed8da1ce0cf129 on tablet 34af2cfd01c7406da5ed8da1ce0cf129 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:37.087075 4412 raft_consensus.cc:1203] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:37.106480 4412 raft_consensus.cc:1203] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:37.115916 4412 raft_consensus.cc:1203] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:37.170212 4412 raft_consensus.cc:1203] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:37.521281 4490 raft_consensus.cc:1050] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:37.522423 4490 consensus_queue.cc:237] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.524816 4412 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.524905 3921 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.525313 4491 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.525745 4464 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.525406 3769 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.526638 4464 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.530706 4491 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.533704 4412 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.535585 3921 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.537482 3669 catalog_manager.cc:5484] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:37.551084 3844 consensus_queue.cc:237] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.553054 3921 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.553212 4412 raft_consensus.cc:1261] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.553681 4490 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.554044 4490 consensus_queue.cc:1035] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.555948 4464 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.556736 3919 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.559706 3655 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1101bf1e3c82448ebf8747b8533fd2f0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:37.562444 3669 catalog_manager.cc:5484] T 1101bf1e3c82448ebf8747b8533fd2f0 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.563257 4412 raft_consensus.cc:2932] T 1101bf1e3c82448ebf8747b8533fd2f0 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.566488 3760 tablet_service.cc:1514] Processing DeleteTablet for tablet 1101bf1e3c82448ebf8747b8533fd2f0 with delete_type TABLET_DATA_TOMBSTONED (TS 6e0d1128f0404d609df72fd607c20650 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58242
I20240503 02:13:37.567478 4500 tablet_replica.cc:330] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: stopping tablet replica
I20240503 02:13:37.567716 4500 raft_consensus.cc:2229] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:37.567925 4500 pending_rounds.cc:62] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Trying to abort 1 pending ops.
I20240503 02:13:37.568248 4500 pending_rounds.cc:68] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023421102169505792 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "1101bf1e3c82448ebf8747b8533fd2f0" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } } }
I20240503 02:13:37.568717 4500 raft_consensus.cc:2894] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20240503 02:13:37.568830 4500 raft_consensus.cc:2866] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240503 02:13:37.568943 4500 raft_consensus.cc:2258] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:37.569914 4500 ts_tablet_manager.cc:1875] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:37.576138 4500 ts_tablet_manager.cc:1888] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:37.576262 4500 log.cc:1196] T 1101bf1e3c82448ebf8747b8533fd2f0 P 6e0d1128f0404d609df72fd607c20650: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root/wals/1101bf1e3c82448ebf8747b8533fd2f0
I20240503 02:13:37.577196 3656 catalog_manager.cc:4830] TS 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): tablet 1101bf1e3c82448ebf8747b8533fd2f0 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:37.596935 4464 raft_consensus.cc:1050] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:37.597709 4464 consensus_queue.cc:237] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.599771 3919 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.599812 3769 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.599993 4412 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.600311 4490 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.600533 4491 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.600898 4463 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.615685 4463 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.616458 4412 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.616968 3919 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.620579 3668 catalog_manager.cc:5484] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.621443 3769 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.629381 4490 raft_consensus.cc:1050] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:37.630216 4490 consensus_queue.cc:237] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.632429 3846 consensus_queue.cc:237] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.634102 3919 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.634102 3921 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.634331 3771 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.634833 4412 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.634786 4463 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.634959 4411 raft_consensus.cc:1261] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.635366 4463 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.635869 4464 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.636054 4507 consensus_queue.cc:1035] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.636683 4463 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.640304 4490 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.641119 4411 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.642095 4507 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.645175 3655 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c1aea07837f41a281d484e37eab5744 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:37.646420 3919 raft_consensus.cc:2932] T 8c1aea07837f41a281d484e37eab5744 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.646803 3920 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.648500 4411 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.650952 3668 catalog_manager.cc:5484] T 8c1aea07837f41a281d484e37eab5744 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:37.652663 3771 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.656113 3760 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c1aea07837f41a281d484e37eab5744 with delete_type TABLET_DATA_TOMBSTONED (TS 6e0d1128f0404d609df72fd607c20650 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58242
I20240503 02:13:37.656991 4500 tablet_replica.cc:330] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: stopping tablet replica
I20240503 02:13:37.657217 4500 raft_consensus.cc:2229] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:37.657507 4500 raft_consensus.cc:2258] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:37.658910 4500 ts_tablet_manager.cc:1875] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:37.659107 3671 catalog_manager.cc:5484] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.667881 3841 consensus_queue.cc:237] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.669425 4411 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.669667 3771 raft_consensus.cc:1261] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.669862 4507 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.670094 4490 consensus_queue.cc:1035] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.670213 4500 ts_tablet_manager.cc:1888] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:37.672123 4490 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.672767 4411 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.675792 3771 raft_consensus.cc:2932] T 34af2cfd01c7406da5ed8da1ce0cf129 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.677181 3655 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 34af2cfd01c7406da5ed8da1ce0cf129 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:37.677855 4489 raft_consensus.cc:1050] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:37.678580 4489 consensus_queue.cc:237] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.679311 3668 catalog_manager.cc:5484] T 34af2cfd01c7406da5ed8da1ce0cf129 P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.682827 3903 tablet_service.cc:1514] Processing DeleteTablet for tablet 34af2cfd01c7406da5ed8da1ce0cf129 with delete_type TABLET_DATA_TOMBSTONED (TS 4001ccb96b064ad2a2ca9c61096516fe not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45608
I20240503 02:13:37.684391 3846 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.685477 4505 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.687776 3771 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.688235 4505 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:37.697610 4512 tablet_replica.cc:330] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: stopping tablet replica
I20240503 02:13:37.697832 4512 raft_consensus.cc:2229] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:37.698185 4512 raft_consensus.cc:2258] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:37.698525 4411 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:37.699431 4512 ts_tablet_manager.cc:1875] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:37.699527 4513 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240503 02:13:37.705292 4512 ts_tablet_manager.cc:1888] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:37.705407 4512 log.cc:1196] T 34af2cfd01c7406da5ed8da1ce0cf129 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root/wals/34af2cfd01c7406da5ed8da1ce0cf129
I20240503 02:13:37.706244 3658 catalog_manager.cc:4830] TS 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): tablet 34af2cfd01c7406da5ed8da1ce0cf129 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:37.711630 4010 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.715535 4500 log.cc:1196] T 8c1aea07837f41a281d484e37eab5744 P 6e0d1128f0404d609df72fd607c20650: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root/wals/8c1aea07837f41a281d484e37eab5744
I20240503 02:13:37.716208 3656 catalog_manager.cc:4830] TS 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): tablet 8c1aea07837f41a281d484e37eab5744 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:37.715997 3846 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.716033 3771 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.717060 3669 catalog_manager.cc:5484] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:37.719961 4411 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.721385 3919 consensus_queue.cc:237] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:37.725064 4412 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.725513 3770 raft_consensus.cc:1261] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:37.726004 4513 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.726567 4505 consensus_queue.cc:1035] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:37.730734 4412 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.731185 4010 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.733808 3771 raft_consensus.cc:2932] T 518cefb57371493496e276f5a10fd53d P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.734656 3658 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 518cefb57371493496e276f5a10fd53d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:37.736105 3669 catalog_manager.cc:5484] T 518cefb57371493496e276f5a10fd53d P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:37.740171 3833 tablet_service.cc:1514] Processing DeleteTablet for tablet 518cefb57371493496e276f5a10fd53d with delete_type TABLET_DATA_TOMBSTONED (TS 2c8f35a9c7534ec2b0251e826a6a9da7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52486
I20240503 02:13:37.740839 4517 tablet_replica.cc:330] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: stopping tablet replica
I20240503 02:13:37.741039 4517 raft_consensus.cc:2229] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:37.741262 4517 raft_consensus.cc:2258] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:37.741895 4517 ts_tablet_manager.cc:1875] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:37.746330 4517 ts_tablet_manager.cc:1888] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:37.746420 4517 log.cc:1196] T 518cefb57371493496e276f5a10fd53d P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root/wals/518cefb57371493496e276f5a10fd53d
I20240503 02:13:37.746999 3655 catalog_manager.cc:4830] TS 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): tablet 518cefb57371493496e276f5a10fd53d (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:38.759698 3771 consensus_queue.cc:237] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:38.762420 3919 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.762465 3841 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.763015 4522 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.763278 4266 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.768880 4266 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.769560 3841 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
W20240503 02:13:38.770274 3736 consensus_peers.cc:485] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 072c60ef4387498abec90d6716f856b8. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:38.775976 3668 catalog_manager.cc:5484] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:38.775951 3770 consensus_queue.cc:237] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
W20240503 02:13:38.783346 3736 consensus_peers.cc:485] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a1c6e46e6b9f47b199a76a6dc02d5a94. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:38.783787 3846 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.784726 3920 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.785586 4266 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.785888 4527 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.787853 3920 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.797161 4530 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.797801 3841 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.797873 3919 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.802774 3770 consensus_queue.cc:237] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:38.804898 3841 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:13:38.805079 3736 consensus_peers.cc:485] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffacc2d4cc534452ad50a87526626004. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:38.805384 4522 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.807058 3919 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.807883 3669 catalog_manager.cc:5484] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:38.808187 4522 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.810652 4266 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.812197 3841 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.815721 3920 consensus_queue.cc:237] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:38.816299 3919 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
W20240503 02:13:38.818593 3885 consensus_peers.cc:485] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3450807dfc52460dbcbbd75dc8b71cb9. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:38.819528 3670 catalog_manager.cc:5484] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.820570 3770 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.822305 4518 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.825420 4010 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.826045 3770 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:38.827553 3841 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:38.828013 4540 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:38.831864 3669 catalog_manager.cc:5484] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:38.834170 3841 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:39.140781 4544 ts_tablet_manager.cc:902] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841)
I20240503 02:13:39.142371 4544 tablet_copy_client.cc:308] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.65:33841
I20240503 02:13:39.149693 3783 tablet_copy_service.cc:140] P 6e0d1128f0404d609df72fd607c20650: Received BeginTabletCopySession request for tablet 072c60ef4387498abec90d6716f856b8 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:42166)
I20240503 02:13:39.149850 3783 tablet_copy_service.cc:161] P 6e0d1128f0404d609df72fd607c20650: Beginning new tablet copy session on tablet 072c60ef4387498abec90d6716f856b8 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:42166: session id = 5be54f33baf0422ab4ebf2f11e01e108-072c60ef4387498abec90d6716f856b8
I20240503 02:13:39.152200 3783 tablet_copy_source_session.cc:205] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:39.153322 4544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072c60ef4387498abec90d6716f856b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:39.158922 4544 tablet_copy_client.cc:783] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:39.159144 4544 tablet_copy_client.cc:647] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:39.161178 4544 tablet_copy_client.cc:523] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:39.164497 4544 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:39.170096 4544 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:39.170500 4544 tablet_bootstrap.cc:492] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:39.170739 4544 ts_tablet_manager.cc:1367] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240503 02:13:39.171480 4544 raft_consensus.cc:348] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.171707 4544 raft_consensus.cc:729] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:39.171916 4544 consensus_queue.cc:260] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.172684 4544 ts_tablet_manager.cc:1398] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:39.173403 3783 tablet_copy_service.cc:342] P 6e0d1128f0404d609df72fd607c20650: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-072c60ef4387498abec90d6716f856b8 received from {username='slave'} at 127.0.0.1:42166
I20240503 02:13:39.173511 3783 tablet_copy_service.cc:433] P 6e0d1128f0404d609df72fd607c20650: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-072c60ef4387498abec90d6716f856b8 on tablet 072c60ef4387498abec90d6716f856b8 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:39.296986 4544 ts_tablet_manager.cc:902] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841)
I20240503 02:13:39.302512 4544 tablet_copy_client.cc:308] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.65:33841
I20240503 02:13:39.303164 3783 tablet_copy_service.cc:140] P 6e0d1128f0404d609df72fd607c20650: Received BeginTabletCopySession request for tablet ffacc2d4cc534452ad50a87526626004 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:42166)
I20240503 02:13:39.303310 3783 tablet_copy_service.cc:161] P 6e0d1128f0404d609df72fd607c20650: Beginning new tablet copy session on tablet ffacc2d4cc534452ad50a87526626004 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:42166: session id = 5be54f33baf0422ab4ebf2f11e01e108-ffacc2d4cc534452ad50a87526626004
I20240503 02:13:39.305786 4556 ts_tablet_manager.cc:902] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841)
I20240503 02:13:39.306087 3783 tablet_copy_source_session.cc:205] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:39.307327 4556 tablet_copy_client.cc:308] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.65:33841
I20240503 02:13:39.307567 4544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffacc2d4cc534452ad50a87526626004. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:39.307936 3783 tablet_copy_service.cc:140] P 6e0d1128f0404d609df72fd607c20650: Received BeginTabletCopySession request for tablet a1c6e46e6b9f47b199a76a6dc02d5a94 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:42166)
I20240503 02:13:39.308033 3783 tablet_copy_service.cc:161] P 6e0d1128f0404d609df72fd607c20650: Beginning new tablet copy session on tablet a1c6e46e6b9f47b199a76a6dc02d5a94 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:42166: session id = 5be54f33baf0422ab4ebf2f11e01e108-a1c6e46e6b9f47b199a76a6dc02d5a94
I20240503 02:13:39.310042 3783 tablet_copy_source_session.cc:205] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:39.310977 4556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1c6e46e6b9f47b199a76a6dc02d5a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:39.312381 4544 tablet_copy_client.cc:783] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:39.312676 4544 tablet_copy_client.cc:647] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:39.314009 4544 tablet_copy_client.cc:523] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:39.316083 4556 tablet_copy_client.cc:783] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:39.316273 4556 tablet_copy_client.cc:647] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:39.317028 4544 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:39.317643 4556 tablet_copy_client.cc:523] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:39.320958 4556 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:39.322332 4544 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:39.322669 4544 tablet_bootstrap.cc:492] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:39.322899 4544 ts_tablet_manager.cc:1367] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.007s sys 0.001s
I20240503 02:13:39.323635 4544 raft_consensus.cc:348] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.323911 4544 raft_consensus.cc:729] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:39.324110 4544 consensus_queue.cc:260] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.324613 4544 ts_tablet_manager.cc:1398] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:13:39.325259 3783 tablet_copy_service.cc:342] P 6e0d1128f0404d609df72fd607c20650: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-ffacc2d4cc534452ad50a87526626004 received from {username='slave'} at 127.0.0.1:42166
I20240503 02:13:39.325383 3783 tablet_copy_service.cc:433] P 6e0d1128f0404d609df72fd607c20650: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-ffacc2d4cc534452ad50a87526626004 on tablet ffacc2d4cc534452ad50a87526626004 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:39.328325 4556 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:39.328744 4556 tablet_bootstrap.cc:492] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:39.329028 4556 ts_tablet_manager.cc:1367] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20240503 02:13:39.329785 4556 raft_consensus.cc:348] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.330024 4556 raft_consensus.cc:729] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:39.330215 4556 consensus_queue.cc:260] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.330881 4556 ts_tablet_manager.cc:1398] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240503 02:13:39.331643 3783 tablet_copy_service.cc:342] P 6e0d1128f0404d609df72fd607c20650: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-a1c6e46e6b9f47b199a76a6dc02d5a94 received from {username='slave'} at 127.0.0.1:42166
I20240503 02:13:39.331763 3783 tablet_copy_service.cc:433] P 6e0d1128f0404d609df72fd607c20650: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-a1c6e46e6b9f47b199a76a6dc02d5a94 on tablet a1c6e46e6b9f47b199a76a6dc02d5a94 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:39.399691 4556 ts_tablet_manager.cc:902] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:39.411216 4556 tablet_copy_client.cc:308] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.67:34983
I20240503 02:13:39.411844 3932 tablet_copy_service.cc:140] P 4001ccb96b064ad2a2ca9c61096516fe: Received BeginTabletCopySession request for tablet 3450807dfc52460dbcbbd75dc8b71cb9 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:46886)
I20240503 02:13:39.411994 3932 tablet_copy_service.cc:161] P 4001ccb96b064ad2a2ca9c61096516fe: Beginning new tablet copy session on tablet 3450807dfc52460dbcbbd75dc8b71cb9 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:46886: session id = 5be54f33baf0422ab4ebf2f11e01e108-3450807dfc52460dbcbbd75dc8b71cb9
I20240503 02:13:39.414252 3932 tablet_copy_source_session.cc:205] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:39.415360 4556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3450807dfc52460dbcbbd75dc8b71cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:39.420655 4556 tablet_copy_client.cc:783] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:39.421200 4556 tablet_copy_client.cc:647] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:39.422562 4556 tablet_copy_client.cc:523] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:39.425441 4556 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:39.430805 4556 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:39.431159 4556 tablet_bootstrap.cc:492] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:39.431377 4556 ts_tablet_manager.cc:1367] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.001s
I20240503 02:13:39.431924 4556 raft_consensus.cc:348] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.432195 4556 raft_consensus.cc:729] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:39.432379 4556 consensus_queue.cc:260] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:39.432775 4556 ts_tablet_manager.cc:1398] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:13:39.433382 3932 tablet_copy_service.cc:342] P 4001ccb96b064ad2a2ca9c61096516fe: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-3450807dfc52460dbcbbd75dc8b71cb9 received from {username='slave'} at 127.0.0.1:46886
I20240503 02:13:39.433497 3932 tablet_copy_service.cc:433] P 4001ccb96b064ad2a2ca9c61096516fe: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-3450807dfc52460dbcbbd75dc8b71cb9 on tablet 3450807dfc52460dbcbbd75dc8b71cb9 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:39.684535 4412 raft_consensus.cc:1203] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:39.748888 4412 raft_consensus.cc:1203] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:39.774685 4412 raft_consensus.cc:1203] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:40.011265 4412 raft_consensus.cc:1203] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:40.107009 4569 raft_consensus.cc:1050] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:40.107860 4569 consensus_queue.cc:237] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.110198 3919 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.110239 4412 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.110767 4570 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.110997 4530 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.111243 3841 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.111831 4570 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.115065 4530 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.119251 4412 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.119295 3920 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.125262 3668 catalog_manager.cc:5484] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.131850 3841 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.138298 3770 consensus_queue.cc:237] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.140251 3920 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.140619 4412 raft_consensus.cc:1261] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.140769 4530 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.141775 4530 consensus_queue.cc:1035] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.143165 4570 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.143715 3919 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.145246 4412 raft_consensus.cc:2932] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.146713 3656 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a1c6e46e6b9f47b199a76a6dc02d5a94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:40.150283 3669 catalog_manager.cc:5484] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:40.153864 3833 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c6e46e6b9f47b199a76a6dc02d5a94 with delete_type TABLET_DATA_TOMBSTONED (TS 2c8f35a9c7534ec2b0251e826a6a9da7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52486
I20240503 02:13:40.154656 4580 tablet_replica.cc:330] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: stopping tablet replica
I20240503 02:13:40.154887 4580 raft_consensus.cc:2229] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:40.155156 4580 raft_consensus.cc:2258] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:40.155926 4580 ts_tablet_manager.cc:1875] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:40.161327 4580 ts_tablet_manager.cc:1888] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:40.161463 4580 log.cc:1196] T a1c6e46e6b9f47b199a76a6dc02d5a94 P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root/wals/a1c6e46e6b9f47b199a76a6dc02d5a94
I20240503 02:13:40.162598 3655 catalog_manager.cc:4830] TS 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): tablet a1c6e46e6b9f47b199a76a6dc02d5a94 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:40.190711 4530 raft_consensus.cc:1050] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:40.191437 4530 consensus_queue.cc:237] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.193699 3920 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.193933 4412 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.194155 4569 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.194635 4570 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.196516 3841 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.203034 4570 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.205370 4582 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.208751 3841 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.208822 3919 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.210786 3668 catalog_manager.cc:5484] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:40.225688 4412 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.226410 3769 consensus_queue.cc:237] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.228013 3841 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.228477 4530 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.230358 4570 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.230820 4411 raft_consensus.cc:1261] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.231451 4582 consensus_queue.cc:1035] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.232591 4411 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.234270 3656 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 072c60ef4387498abec90d6716f856b8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:40.234369 3846 raft_consensus.cc:2932] T 072c60ef4387498abec90d6716f856b8 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.238193 3668 catalog_manager.cc:5484] T 072c60ef4387498abec90d6716f856b8 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:40.241569 3903 tablet_service.cc:1514] Processing DeleteTablet for tablet 072c60ef4387498abec90d6716f856b8 with delete_type TABLET_DATA_TOMBSTONED (TS 4001ccb96b064ad2a2ca9c61096516fe not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45608
I20240503 02:13:40.242136 4586 tablet_replica.cc:330] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: stopping tablet replica
I20240503 02:13:40.242316 4586 raft_consensus.cc:2229] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:40.242545 4586 raft_consensus.cc:2258] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:40.244320 4586 ts_tablet_manager.cc:1875] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:40.249148 4586 ts_tablet_manager.cc:1888] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:40.249261 4586 log.cc:1196] T 072c60ef4387498abec90d6716f856b8 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root/wals/072c60ef4387498abec90d6716f856b8
I20240503 02:13:40.249850 3658 catalog_manager.cc:4830] TS 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): tablet 072c60ef4387498abec90d6716f856b8 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:40.323678 4582 raft_consensus.cc:1050] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:40.324481 4582 consensus_queue.cc:237] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.326864 4411 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.327368 4570 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.327477 3920 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.327839 3846 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.328219 4570 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.328450 4569 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.332072 4570 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.332752 3919 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.333283 3846 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.336635 4411 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.338624 3669 catalog_manager.cc:5484] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:40.344471 3769 consensus_queue.cc:237] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.346334 4411 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.347870 4569 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.348691 3846 raft_consensus.cc:1261] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.349200 4582 consensus_queue.cc:1035] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.350504 4569 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.351136 4411 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.352274 3846 raft_consensus.cc:2932] T ffacc2d4cc534452ad50a87526626004 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.355057 3656 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ffacc2d4cc534452ad50a87526626004 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:40.356765 3670 catalog_manager.cc:5484] T ffacc2d4cc534452ad50a87526626004 P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.361841 3903 tablet_service.cc:1514] Processing DeleteTablet for tablet ffacc2d4cc534452ad50a87526626004 with delete_type TABLET_DATA_TOMBSTONED (TS 4001ccb96b064ad2a2ca9c61096516fe not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45608
I20240503 02:13:40.362221 4586 tablet_replica.cc:330] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: stopping tablet replica
I20240503 02:13:40.362449 4586 raft_consensus.cc:2229] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:40.362661 4586 raft_consensus.cc:2258] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:40.363489 4586 ts_tablet_manager.cc:1875] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:40.369503 4586 ts_tablet_manager.cc:1888] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:40.369648 4586 log.cc:1196] T ffacc2d4cc534452ad50a87526626004 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root/wals/ffacc2d4cc534452ad50a87526626004
I20240503 02:13:40.370385 3658 catalog_manager.cc:4830] TS 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): tablet ffacc2d4cc534452ad50a87526626004 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:40.411034 4518 raft_consensus.cc:1050] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:40.411720 4518 consensus_queue.cc:237] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.414420 3769 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.414682 4411 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.414945 4571 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.415079 3846 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:40.415639 4563 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.416132 4563 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:40.428037 4563 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.430310 3769 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.429435 3846 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.433295 4411 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.434232 3669 catalog_manager.cc:5484] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:40.442746 3919 consensus_queue.cc:237] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:40.444568 4411 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.445171 4518 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.446101 3769 raft_consensus.cc:1261] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:40.446611 4592 consensus_queue.cc:1035] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:40.447983 4518 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.448577 4411 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.450829 3658 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3450807dfc52460dbcbbd75dc8b71cb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:40.451346 3769 raft_consensus.cc:2932] T 3450807dfc52460dbcbbd75dc8b71cb9 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.453822 3671 catalog_manager.cc:5484] T 3450807dfc52460dbcbbd75dc8b71cb9 P 5be54f33baf0422ab4ebf2f11e01e108 reported cstate change: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:40.458756 3833 tablet_service.cc:1514] Processing DeleteTablet for tablet 3450807dfc52460dbcbbd75dc8b71cb9 with delete_type TABLET_DATA_TOMBSTONED (TS 2c8f35a9c7534ec2b0251e826a6a9da7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52486
I20240503 02:13:40.459017 4580 tablet_replica.cc:330] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: stopping tablet replica
I20240503 02:13:40.459218 4580 raft_consensus.cc:2229] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:40.459448 4580 raft_consensus.cc:2258] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:40.460254 4580 ts_tablet_manager.cc:1875] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:40.464502 4580 ts_tablet_manager.cc:1888] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:40.464584 4580 log.cc:1196] T 3450807dfc52460dbcbbd75dc8b71cb9 P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root/wals/3450807dfc52460dbcbbd75dc8b71cb9
I20240503 02:13:40.465224 3655 catalog_manager.cc:4830] TS 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): tablet 3450807dfc52460dbcbbd75dc8b71cb9 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:41.471324 3841 consensus_queue.cc:237] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.473529 3769 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:13:41.473592 3811 consensus_peers.cc:485] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4094a9a8c23046b6a2f27c451d8347f3. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:41.473572 3919 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:41.474097 4507 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.474395 4597 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.477038 4490 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.477808 3769 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.477906 3919 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.483386 3671 catalog_manager.cc:5484] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:41.485145 3841 consensus_queue.cc:237] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.494467 3919 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:13:41.495206 3811 consensus_peers.cc:485] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 353eaefa93a64cd19b50fcfb916bfed6. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:41.495646 4490 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.496915 3769 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:41.497577 4605 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.500177 4597 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.500838 3919 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.501147 3769 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.505753 3920 consensus_queue.cc:237] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.506179 3669 catalog_manager.cc:5484] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.508539 3841 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:41.509033 4563 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240503 02:13:41.509289 3885 consensus_peers.cc:485] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de6622d0a3f54d418d588fe0281d9324. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:41.510383 3769 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:41.511392 4609 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.512039 4592 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.512358 3841 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.517112 3769 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.518508 3844 consensus_queue.cc:237] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.520743 3671 catalog_manager.cc:5484] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
W20240503 02:13:41.524454 3811 consensus_peers.cc:485] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88bc3ed40ce7448a98251d25e2a09798. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:41.524456 3769 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:41.525009 3920 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:41.525837 4605 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.526088 4490 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:41.529136 4490 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.532184 3920 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.538576 3669 catalog_manager.cc:5484] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:41.547245 3769 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:41.900246 4616 ts_tablet_manager.cc:902] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:41.902164 4616 tablet_copy_client.cc:308] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.66:43979
I20240503 02:13:41.902657 3858 tablet_copy_service.cc:140] P 2c8f35a9c7534ec2b0251e826a6a9da7: Received BeginTabletCopySession request for tablet 4094a9a8c23046b6a2f27c451d8347f3 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:43200)
I20240503 02:13:41.902791 3858 tablet_copy_service.cc:161] P 2c8f35a9c7534ec2b0251e826a6a9da7: Beginning new tablet copy session on tablet 4094a9a8c23046b6a2f27c451d8347f3 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:43200: session id = 5be54f33baf0422ab4ebf2f11e01e108-4094a9a8c23046b6a2f27c451d8347f3
I20240503 02:13:41.904881 3858 tablet_copy_source_session.cc:205] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:41.905946 4616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4094a9a8c23046b6a2f27c451d8347f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:41.911442 4616 tablet_copy_client.cc:783] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:41.911662 4616 tablet_copy_client.cc:647] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:41.913055 4616 tablet_copy_client.cc:523] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:41.915794 4616 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:41.921012 4616 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:41.921414 4616 tablet_bootstrap.cc:492] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:41.921690 4616 ts_tablet_manager.cc:1367] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.007s sys 0.001s
I20240503 02:13:41.922376 4616 raft_consensus.cc:348] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.922557 4616 raft_consensus.cc:729] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:41.922689 4616 consensus_queue.cc:260] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.923385 4616 ts_tablet_manager.cc:1398] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:13:41.924618 3858 tablet_copy_service.cc:342] P 2c8f35a9c7534ec2b0251e826a6a9da7: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-4094a9a8c23046b6a2f27c451d8347f3 received from {username='slave'} at 127.0.0.1:43200
I20240503 02:13:41.924695 3858 tablet_copy_service.cc:433] P 2c8f35a9c7534ec2b0251e826a6a9da7: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-4094a9a8c23046b6a2f27c451d8347f3 on tablet 4094a9a8c23046b6a2f27c451d8347f3 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:41.941121 4616 ts_tablet_manager.cc:902] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
I20240503 02:13:41.943624 4616 tablet_copy_client.cc:308] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.67:34983
I20240503 02:13:41.944895 3932 tablet_copy_service.cc:140] P 4001ccb96b064ad2a2ca9c61096516fe: Received BeginTabletCopySession request for tablet de6622d0a3f54d418d588fe0281d9324 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:46886)
I20240503 02:13:41.945029 3932 tablet_copy_service.cc:161] P 4001ccb96b064ad2a2ca9c61096516fe: Beginning new tablet copy session on tablet de6622d0a3f54d418d588fe0281d9324 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:46886: session id = 5be54f33baf0422ab4ebf2f11e01e108-de6622d0a3f54d418d588fe0281d9324
I20240503 02:13:41.947408 3932 tablet_copy_source_session.cc:205] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:41.948419 4616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6622d0a3f54d418d588fe0281d9324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:41.954043 4616 tablet_copy_client.cc:783] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:41.954211 4616 tablet_copy_client.cc:647] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:41.955684 4616 tablet_copy_client.cc:523] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:41.958099 4616 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:41.962221 4616 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:41.962554 4616 tablet_bootstrap.cc:492] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:41.962752 4616 ts_tablet_manager.cc:1367] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20240503 02:13:41.963340 4616 raft_consensus.cc:348] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.963498 4616 raft_consensus.cc:729] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:41.963685 4616 consensus_queue.cc:260] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:41.964133 4616 ts_tablet_manager.cc:1398] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240503 02:13:41.964771 3932 tablet_copy_service.cc:342] P 4001ccb96b064ad2a2ca9c61096516fe: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-de6622d0a3f54d418d588fe0281d9324 received from {username='slave'} at 127.0.0.1:46886
I20240503 02:13:41.964872 3932 tablet_copy_service.cc:433] P 4001ccb96b064ad2a2ca9c61096516fe: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-de6622d0a3f54d418d588fe0281d9324 on tablet de6622d0a3f54d418d588fe0281d9324 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:42.090898 4616 ts_tablet_manager.cc:902] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:42.092235 4616 tablet_copy_client.cc:308] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.66:43979
I20240503 02:13:42.092806 3858 tablet_copy_service.cc:140] P 2c8f35a9c7534ec2b0251e826a6a9da7: Received BeginTabletCopySession request for tablet 353eaefa93a64cd19b50fcfb916bfed6 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:43200)
I20240503 02:13:42.092918 3858 tablet_copy_service.cc:161] P 2c8f35a9c7534ec2b0251e826a6a9da7: Beginning new tablet copy session on tablet 353eaefa93a64cd19b50fcfb916bfed6 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:43200: session id = 5be54f33baf0422ab4ebf2f11e01e108-353eaefa93a64cd19b50fcfb916bfed6
I20240503 02:13:42.095394 3858 tablet_copy_source_session.cc:205] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:42.096813 4616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353eaefa93a64cd19b50fcfb916bfed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:42.101728 4616 tablet_copy_client.cc:783] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:42.101945 4616 tablet_copy_client.cc:647] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:42.103318 4616 tablet_copy_client.cc:523] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:42.106256 4616 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:42.111933 4616 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:42.112274 4616 tablet_bootstrap.cc:492] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:42.112498 4616 ts_tablet_manager.cc:1367] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240503 02:13:42.113147 4616 raft_consensus.cc:348] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:42.113385 4616 raft_consensus.cc:729] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:42.113592 4616 consensus_queue.cc:260] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:42.114074 4616 ts_tablet_manager.cc:1398] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:13:42.116031 3858 tablet_copy_service.cc:342] P 2c8f35a9c7534ec2b0251e826a6a9da7: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-353eaefa93a64cd19b50fcfb916bfed6 received from {username='slave'} at 127.0.0.1:43200
I20240503 02:13:42.116127 3858 tablet_copy_service.cc:433] P 2c8f35a9c7534ec2b0251e826a6a9da7: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-353eaefa93a64cd19b50fcfb916bfed6 on tablet 353eaefa93a64cd19b50fcfb916bfed6 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:42.129623 4616 ts_tablet_manager.cc:902] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979)
I20240503 02:13:42.131186 4616 tablet_copy_client.cc:308] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Beginning tablet copy session from remote peer at address 127.2.157.66:43979
I20240503 02:13:42.131758 3858 tablet_copy_service.cc:140] P 2c8f35a9c7534ec2b0251e826a6a9da7: Received BeginTabletCopySession request for tablet 88bc3ed40ce7448a98251d25e2a09798 from peer 5be54f33baf0422ab4ebf2f11e01e108 ({username='slave'} at 127.0.0.1:43200)
I20240503 02:13:42.131870 3858 tablet_copy_service.cc:161] P 2c8f35a9c7534ec2b0251e826a6a9da7: Beginning new tablet copy session on tablet 88bc3ed40ce7448a98251d25e2a09798 from peer 5be54f33baf0422ab4ebf2f11e01e108 at {username='slave'} at 127.0.0.1:43200: session id = 5be54f33baf0422ab4ebf2f11e01e108-88bc3ed40ce7448a98251d25e2a09798
I20240503 02:13:42.134177 3858 tablet_copy_source_session.cc:205] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:13:42.135284 4616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bc3ed40ce7448a98251d25e2a09798. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:13:42.141422 4616 tablet_copy_client.cc:783] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 0 data blocks...
I20240503 02:13:42.141656 4616 tablet_copy_client.cc:647] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Starting download of 1 WAL segments...
I20240503 02:13:42.143469 4616 tablet_copy_client.cc:523] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:13:42.146284 4616 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap starting.
I20240503 02:13:42.152226 4616 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:13:42.152480 4616 tablet_bootstrap.cc:492] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: Bootstrap complete.
I20240503 02:13:42.152688 4616 ts_tablet_manager.cc:1367] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20240503 02:13:42.153179 4616 raft_consensus.cc:348] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:42.153395 4616 raft_consensus.cc:729] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5be54f33baf0422ab4ebf2f11e01e108, State: Initialized, Role: LEARNER
I20240503 02:13:42.153596 4616 consensus_queue.cc:260] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:42.154400 4616 ts_tablet_manager.cc:1398] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240503 02:13:42.155284 3858 tablet_copy_service.cc:342] P 2c8f35a9c7534ec2b0251e826a6a9da7: Request end of tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-88bc3ed40ce7448a98251d25e2a09798 received from {username='slave'} at 127.0.0.1:43200
I20240503 02:13:42.155360 3858 tablet_copy_service.cc:433] P 2c8f35a9c7534ec2b0251e826a6a9da7: ending tablet copy session 5be54f33baf0422ab4ebf2f11e01e108-88bc3ed40ce7448a98251d25e2a09798 on tablet 88bc3ed40ce7448a98251d25e2a09798 with peer 5be54f33baf0422ab4ebf2f11e01e108
I20240503 02:13:42.325325 4412 raft_consensus.cc:1203] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:42.334070 4412 raft_consensus.cc:1203] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:42.582222 4412 raft_consensus.cc:1203] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:42.620831 4412 raft_consensus.cc:1203] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240503 02:13:42.727859 4639 raft_consensus.cc:1050] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:42.728524 4639 consensus_queue.cc:237] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:42.730521 4412 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:42.730788 3920 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:42.730926 3769 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:42.731037 4605 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:42.731567 4605 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:42.732697 4490 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:42.741163 4639 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.745456 4412 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.748152 3669 catalog_manager.cc:5484] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:42.750627 3769 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.744699 3920 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.771066 3844 consensus_queue.cc:237] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:42.773435 4412 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:42.773756 3920 raft_consensus.cc:1261] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:42.773962 4645 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:42.774206 4490 consensus_queue.cc:1035] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:42.776929 4639 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.777523 4412 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.780057 3919 raft_consensus.cc:2932] T 4094a9a8c23046b6a2f27c451d8347f3 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.780751 3655 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4094a9a8c23046b6a2f27c451d8347f3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:42.781739 3669 catalog_manager.cc:5484] T 4094a9a8c23046b6a2f27c451d8347f3 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:42.785624 3760 tablet_service.cc:1514] Processing DeleteTablet for tablet 4094a9a8c23046b6a2f27c451d8347f3 with delete_type TABLET_DATA_TOMBSTONED (TS 6e0d1128f0404d609df72fd607c20650 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58242
I20240503 02:13:42.786429 4650 tablet_replica.cc:330] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: stopping tablet replica
I20240503 02:13:42.786597 4650 raft_consensus.cc:2229] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:42.786801 4650 raft_consensus.cc:2258] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:42.787449 4650 ts_tablet_manager.cc:1875] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:42.791916 4650 ts_tablet_manager.cc:1888] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:42.792033 4650 log.cc:1196] T 4094a9a8c23046b6a2f27c451d8347f3 P 6e0d1128f0404d609df72fd607c20650: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root/wals/4094a9a8c23046b6a2f27c451d8347f3
I20240503 02:13:42.792673 3656 catalog_manager.cc:4830] TS 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): tablet 4094a9a8c23046b6a2f27c451d8347f3 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:42.946218 4592 raft_consensus.cc:1050] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:42.946924 4592 consensus_queue.cc:237] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:42.950094 4412 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:42.950284 3846 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:42.950757 4653 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:42.951211 3769 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:42.951570 4592 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:42.952057 4653 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:42.955850 4592 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.956544 3769 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.961938 3670 catalog_manager.cc:5484] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.962293 3846 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.967801 4412 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.971653 3920 consensus_queue.cc:237] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:42.973781 3769 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:42.974385 4653 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:42.974516 4412 raft_consensus.cc:1261] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:42.974967 4651 consensus_queue.cc:1035] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:42.976918 4592 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.977742 3770 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.977977 4412 raft_consensus.cc:2932] T de6622d0a3f54d418d588fe0281d9324 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:42.979990 3658 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de6622d0a3f54d418d588fe0281d9324 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:42.983214 3671 catalog_manager.cc:5484] T de6622d0a3f54d418d588fe0281d9324 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index 3 to 4, VOTER 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66) evicted. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:42.987207 3833 tablet_service.cc:1514] Processing DeleteTablet for tablet de6622d0a3f54d418d588fe0281d9324 with delete_type TABLET_DATA_TOMBSTONED (TS 2c8f35a9c7534ec2b0251e826a6a9da7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52486
I20240503 02:13:42.988135 4658 tablet_replica.cc:330] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: stopping tablet replica
I20240503 02:13:42.988464 4658 raft_consensus.cc:2229] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:42.989840 4658 raft_consensus.cc:2258] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:42.991463 4658 ts_tablet_manager.cc:1875] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:42.996958 4658 ts_tablet_manager.cc:1888] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:42.997071 4658 log.cc:1196] T de6622d0a3f54d418d588fe0281d9324 P 2c8f35a9c7534ec2b0251e826a6a9da7: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-1-root/wals/de6622d0a3f54d418d588fe0281d9324
I20240503 02:13:42.997788 3655 catalog_manager.cc:4830] TS 2c8f35a9c7534ec2b0251e826a6a9da7 (127.2.157.66:43979): tablet de6622d0a3f54d418d588fe0281d9324 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:43.139863 4639 raft_consensus.cc:1050] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:43.140656 4639 consensus_queue.cc:237] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:43.143075 3770 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:43.143218 3920 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:43.143395 4412 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:43.143819 4645 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:43.144366 4645 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:43.145249 4605 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:43.148209 4645 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.148694 3920 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.151870 4412 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.153700 3669 catalog_manager.cc:5484] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:43.156258 3771 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.163779 4645 raft_consensus.cc:1050] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7: attempting to promote NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 to VOTER
I20240503 02:13:43.164566 4645 consensus_queue.cc:237] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:43.165113 3846 consensus_queue.cc:237] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:43.167258 4412 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 LEARNER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:43.167675 4411 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:43.167810 3919 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:43.168311 3920 raft_consensus.cc:1261] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:43.168347 4644 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:43.168522 4639 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:43.169173 3770 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240503 02:13:43.169338 4644 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:43.169903 4639 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:43.170133 4644 consensus_queue.cc:1035] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:43.171528 4490 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.174245 3919 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.175736 4412 raft_consensus.cc:2932] T 353eaefa93a64cd19b50fcfb916bfed6 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.180053 3655 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 353eaefa93a64cd19b50fcfb916bfed6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:43.181582 3670 catalog_manager.cc:5484] T 353eaefa93a64cd19b50fcfb916bfed6 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 3 to 4, VOTER 6e0d1128f0404d609df72fd607c20650 (127.2.157.65) evicted. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:43.184648 4644 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.186925 4411 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.187427 3760 tablet_service.cc:1514] Processing DeleteTablet for tablet 353eaefa93a64cd19b50fcfb916bfed6 with delete_type TABLET_DATA_TOMBSTONED (TS 6e0d1128f0404d609df72fd607c20650 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58242
I20240503 02:13:43.187719 4650 tablet_replica.cc:330] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: stopping tablet replica
I20240503 02:13:43.187901 4650 raft_consensus.cc:2229] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:43.188112 4650 raft_consensus.cc:2258] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:43.189123 4650 ts_tablet_manager.cc:1875] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:43.191135 3919 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.190089 3770 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.197024 3671 catalog_manager.cc:5484] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index 2 to 3, 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.203804 4650 ts_tablet_manager.cc:1888] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:43.203924 4650 log.cc:1196] T 353eaefa93a64cd19b50fcfb916bfed6 P 6e0d1128f0404d609df72fd607c20650: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-0-root/wals/353eaefa93a64cd19b50fcfb916bfed6
I20240503 02:13:43.204634 3656 catalog_manager.cc:4830] TS 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841): tablet 353eaefa93a64cd19b50fcfb916bfed6 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:43.204582 3846 consensus_queue.cc:237] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } }
I20240503 02:13:43.206422 3770 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:43.206475 4411 raft_consensus.cc:1261] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Refusing update from remote peer 2c8f35a9c7534ec2b0251e826a6a9da7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240503 02:13:43.206857 4645 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:43.207114 4639 consensus_queue.cc:1035] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240503 02:13:43.209390 4639 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.209909 4411 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 5be54f33baf0422ab4ebf2f11e01e108 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.210937 3770 raft_consensus.cc:2932] T 88bc3ed40ce7448a98251d25e2a09798 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } } }
I20240503 02:13:43.213761 3655 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 88bc3ed40ce7448a98251d25e2a09798 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240503 02:13:43.215896 3669 catalog_manager.cc:5484] T 88bc3ed40ce7448a98251d25e2a09798 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index 3 to 4, VOTER 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67) evicted. New cstate: current_term: 1 leader_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240503 02:13:43.219720 3903 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bc3ed40ce7448a98251d25e2a09798 with delete_type TABLET_DATA_TOMBSTONED (TS 4001ccb96b064ad2a2ca9c61096516fe not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45608
I20240503 02:13:43.220362 4667 tablet_replica.cc:330] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: stopping tablet replica
I20240503 02:13:43.220561 4667 raft_consensus.cc:2229] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:13:43.220795 4667 raft_consensus.cc:2258] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:13:43.221603 4667 ts_tablet_manager.cc:1875] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:13:43.226233 4667 ts_tablet_manager.cc:1888] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240503 02:13:43.226331 4667 log.cc:1196] T 88bc3ed40ce7448a98251d25e2a09798 P 4001ccb96b064ad2a2ca9c61096516fe: Deleting WAL directory at /tmp/dist-test-task35ZQtY/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1714702355592969-2677-0/minicluster-data/ts-2-root/wals/88bc3ed40ce7448a98251d25e2a09798
I20240503 02:13:43.226929 3658 catalog_manager.cc:4830] TS 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983): tablet 88bc3ed40ce7448a98251d25e2a09798 (table test-workload [id=c49adf327c244a9aa46bc38493fc2805]) successfully deleted
I20240503 02:13:44.230581 3919 consensus_queue.cc:237] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:44.232615 3770 raft_consensus.cc:1261] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:44.232919 3846 raft_consensus.cc:1261] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:44.233050 4651 consensus_queue.cc:1035] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:44.234481 4670 consensus_queue.cc:1035] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240503 02:13:44.235189 3885 consensus_peers.cc:485] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89de6eace7d643e2a78385a67c771991. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:44.236940 4651 raft_consensus.cc:2932] T 89de6eace7d643e2a78385a67c771991 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.237483 3770 raft_consensus.cc:2932] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.238400 3846 raft_consensus.cc:2932] T 89de6eace7d643e2a78385a67c771991 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.243283 3669 catalog_manager.cc:5484] T 89de6eace7d643e2a78385a67c771991 P 6e0d1128f0404d609df72fd607c20650 reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } attrs { replace: true } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.244889 3771 consensus_queue.cc:237] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:44.251016 3846 raft_consensus.cc:1261] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 3 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20240503 02:13:44.251052 3736 consensus_peers.cc:485] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d43c2f4090f14b4baaf2a55a857ed245. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:44.251503 4638 consensus_queue.cc:1035] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:44.251777 3919 raft_consensus.cc:1261] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20240503 02:13:44.252236 4638 consensus_queue.cc:1035] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240503 02:13:44.256572 3846 raft_consensus.cc:2932] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.256964 3919 raft_consensus.cc:2932] T d43c2f4090f14b4baaf2a55a857ed245 P 4001ccb96b064ad2a2ca9c61096516fe [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.256101 4680 raft_consensus.cc:2932] T d43c2f4090f14b4baaf2a55a857ed245 P 6e0d1128f0404d609df72fd607c20650 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.261744 3671 catalog_manager.cc:5484] T d43c2f4090f14b4baaf2a55a857ed245 P 2c8f35a9c7534ec2b0251e826a6a9da7 reported cstate change: config changed from index -1 to 3, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 3 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.264906 3771 consensus_queue.cc:237] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:44.276100 3841 raft_consensus.cc:1261] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:44.276193 3919 raft_consensus.cc:1261] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Refusing update from remote peer 6e0d1128f0404d609df72fd607c20650: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:13:44.276389 3736 consensus_peers.cc:485] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a511b3bba2024783a999d7c7b616100e. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:44.276698 4681 consensus_queue.cc:1035] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:44.276947 4638 consensus_queue.cc:1035] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:44.279176 4681 raft_consensus.cc:2932] T a511b3bba2024783a999d7c7b616100e P 6e0d1128f0404d609df72fd607c20650 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.280171 3919 raft_consensus.cc:2932] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.282618 3841 raft_consensus.cc:2932] T a511b3bba2024783a999d7c7b616100e P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.285244 3669 catalog_manager.cc:5484] T a511b3bba2024783a999d7c7b616100e P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "6e0d1128f0404d609df72fd607c20650" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } attrs { replace: true } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.287204 3919 consensus_queue.cc:237] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } }
I20240503 02:13:44.289407 3841 raft_consensus.cc:1261] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:13:44.289628 3885 consensus_peers.cc:485] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe -> Peer 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68:41241): Couldn't send request to peer 5be54f33baf0422ab4ebf2f11e01e108. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ea517f608bc44a3a34d6fa0fa37e147. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:13:44.289886 4670 consensus_queue.cc:1035] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:44.290567 3771 raft_consensus.cc:1261] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Refusing update from remote peer 4001ccb96b064ad2a2ca9c61096516fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:13:44.291015 4670 consensus_queue.cc:1035] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240503 02:13:44.293299 4651 raft_consensus.cc:2932] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.298913 3771 raft_consensus.cc:2932] T 0ea517f608bc44a3a34d6fa0fa37e147 P 6e0d1128f0404d609df72fd607c20650 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.299553 3841 raft_consensus.cc:2932] T 0ea517f608bc44a3a34d6fa0fa37e147 P 2c8f35a9c7534ec2b0251e826a6a9da7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } } }
I20240503 02:13:44.300063 3670 catalog_manager.cc:5484] T 0ea517f608bc44a3a34d6fa0fa37e147 P 4001ccb96b064ad2a2ca9c61096516fe reported cstate change: config changed from index -1 to 2, NON_VOTER 5be54f33baf0422ab4ebf2f11e01e108 (127.2.157.68) added. New cstate: current_term: 1 leader_uuid: "4001ccb96b064ad2a2ca9c61096516fe" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e0d1128f0404d609df72fd607c20650" member_type: VOTER last_known_addr { host: "127.2.157.65" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c8f35a9c7534ec2b0251e826a6a9da7" member_type: VOTER last_known_addr { host: "127.2.157.66" port: 43979 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4001ccb96b064ad2a2ca9c61096516fe" member_type: VOTER last_known_addr { host: "127.2.157.67" port: 34983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be54f33baf0422ab4ebf2f11e01e108" member_type: NON_VOTER last_known_addr { host: "127.2.157.68" port: 41241 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240503 02:13:44.748553 4688 ts_tablet_manager.cc:902] T 0ea517f608bc44a3a34d6fa0fa37e147 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
F20240503 02:13:44.749056 4688 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (init.queue_time_us, init.queue_time_us) (init.run_cpu_time_us, init.run_cpu_time_us) (init.run_wall_time_us, init.run_wall_time_us) (leader-initialization.queue_time_us, leader-initialization.queue_time_us) (leader-initialization.run_cpu_time_us, leader-initialization.run_cpu_time_us) (leader-initialization.run_wall_time_us, leader-initialization.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-072c60ef4387498abec90d6716f856b8.queue_time_us, tablet-download-pool-072c60ef4387498abec90d6716f856b8.queue_time_us) (tablet-download-pool-072c60ef4387498abec90d6716f856b8.run_cpu_time_us, tablet-download-pool-072c60ef4387498abec90d6716f856b8.run_cpu_time_us) (tablet-download-pool-072c60ef4387498abec90d6716f856b8.run_wall_time_us, tablet-download-pool-072c60ef4387498abec90d6716f856b8.run_wall_time_us) (tablet-download-pool-0ea517f608bc44a3a34d6fa0fa37e147.queue_time_us, tablet-download-pool-0ea517f608bc44a3a34d6fa0fa37e147.queue_time_us) (tablet-download-pool-1101bf1e3c82448ebf8747b8533fd2f0.queue_time_us, tablet-download-pool-1101bf1e3c82448ebf8747b8533fd2f0.queue_time_us) (tablet-download-pool-1101bf1e3c82448ebf8747b8533fd2f0.run_cpu_time_us, tablet-download-pool-1101bf1e3c82448ebf8747b8533fd2f0.run_cpu_time_us) (tablet-download-pool-1101bf1e3c82448ebf8747b8533fd2f0.run_wall_time_us, tablet-download-pool-1101bf1e3c82448ebf8747b8533fd2f0.run_wall_time_us) (tablet-download-pool-3450807dfc52460dbcbbd75dc8b71cb9.queue_time_us, tablet-download-pool-3450807dfc52460dbcbbd75dc8b71cb9.queue_time_us) (tablet-download-pool-3450807dfc52460dbcbbd75dc8b71cb9.run_cpu_time_us, tablet-download-pool-3450807dfc52460dbcbbd75dc8b71cb9.run_cpu_time_us) (tablet-download-pool-3450807dfc52460dbcbbd75dc8b71cb9.run_wall_time_us, tablet-download-pool-3450807dfc52460dbcbbd75dc8b71cb9.run_wall_time_us) (tablet-download-pool-34af2cfd01c7406da5ed8da1ce0cf129.queue_time_us, tablet-download-pool-34af2cfd01c7406da5ed8da1ce0cf129.queue_time_us) (tablet-download-pool-34af2cfd01c7406da5ed8da1ce0cf129.run_cpu_time_us, tablet-download-pool-34af2cfd01c7406da5ed8da1ce0cf129.run_cpu_time_us) (tablet-download-pool-34af2cfd01c7406da5ed8da1ce0cf129.run_wall_time_us, tablet-download-pool-34af2cfd01c7406da5ed8da1ce0cf129.run_wall_time_us) (tablet-download-pool-353eaefa93a64cd19b50fcfb916bfed6.queue_time_us, tablet-download-pool-353eaefa93a64cd19b50fcfb916bfed6.queue_time_us) (tablet-download-pool-353eaefa93a64cd19b50fcfb916bfed6.run_cpu_time_us, tablet-download-pool-353eaefa93a64cd19b50fcfb916bfed6.run_cpu_time_us) (tablet-download-pool-353eaefa93a64cd19b50fcfb916bfed6.run_wall_time_us, tablet-download-pool-353eaefa93a64cd19b50fcfb916bfed6.run_wall_time_us) (tablet-download-pool-4094a9a8c23046b6a2f27c451d8347f3.queue_time_us, tablet-download-pool-4094a9a8c23046b6a2f27c451d8347f3.queue_time_us) (tablet-download-pool-4094a9a8c23046b6a2f27c451d8347f3.run_cpu_time_us, tablet-download-pool-4094a9a8c23046b6a2f27c451d8347f3.run_cpu_time_us) (tablet-download-pool-4094a9a8c23046b6a2f27c451d8347f3.run_wall_time_us, tablet-download-pool-4094a9a8c23046b6a2f27c451d8347f3.run_wall_time_us) (tablet-download-pool-518cefb57371493496e276f5a10fd53d.queue_time_us, tablet-download-pool-518cefb57371493496e276f5a10fd53d.queue_time_us) (tablet-download-pool-518cefb57371493496e276f5a10fd53d.run_cpu_time_us, tablet-download-pool-518cefb57371493496e276f5a10fd53d.run_cpu_time_us) (tablet-download-pool-518cefb57371493496e276f5a10fd53d.run_wall_time_us, tablet-download-pool-518cefb57371493496e276f5a10fd53d.run_wall_time_us) (tablet-download-pool-88bc3ed40ce7448a98251d25e2a09798.queue_time_us, tablet-download-pool-88bc3ed40ce7448a98251d25e2a09798.queue_time_us) (tablet-download-pool-88bc3ed40ce7448a98251d25e2a09798.run_cpu_time_us, tablet-download-pool-88bc3ed40ce7448a98251d25e2a09798.run_cpu_time_us) (tablet-download-pool-88bc3ed40ce7448a98251d25e2a09798.run_wall_time_us, tablet-download-pool-88bc3ed40ce7448a98251d25e2a09798.run_wall_time_us) (tablet-download-pool-8c1aea07837f41a281d484e37eab5744.queue_time_us, tablet-download-pool-8c1aea07837f41a281d484e37eab5744.queue_time_us) (tablet-download-pool-8c1aea07837f41a281d484e37eab5744.run_cpu_time_us, tablet-download-pool-8c1aea07837f41a281d484e37eab5744.run_cpu_time_us) (tablet-download-pool-8c1aea07837f41a281d484e37eab5744.run_wall_time_us, tablet-download-pool-8c1aea07837f41a281d484e37eab5744.run_wall_time_us) (tablet-download-pool-a1c6e46e6b9f47b199a76a6dc02d5a94.queue_time_us, tablet-download-pool-a1c6e46e6b9f47b199a76a6dc02d5a94.queue_time_us) (tablet-download-pool-a1c6e46e6b9f47b199a76a6dc02d5a94.run_cpu_time_us, tablet-download-pool-a1c6e46e6b9f47b199a76a6dc02d5a94.run_cpu_time_us) (tablet-download-pool-a1c6e46e6b9f47b199a76a6dc02d5a94.run_wall_time_us, tablet-download-pool-a1c6e46e6b9f47b199a76a6dc02d5a94.run_wall_time_us) (tablet-download-pool-de6622d0a3f54d418d588fe0281d9324.queue_time_us, tablet-download-pool-de6622d0a3f54d418d588fe0281d9324.queue_time_us) (tablet-download-pool-de6622d0a3f54d418d588fe0281d9324.run_cpu_time_us, tablet-download-pool-de6622d0a3f54d418d588fe0281d9324.run_cpu_time_us) (tablet-download-pool-de6622d0a3f54d418d588fe0281d9324.run_wall_time_us, tablet-download-pool-de6622d0a3f54d418d588fe0281d9324.run_wall_time_us) (tablet-download-pool-ffacc2d4cc534452ad50a87526626004.queue_time_us, tablet-download-pool-ffacc2d4cc534452ad50a87526626004.queue_time_us) (tablet-download-pool-ffacc2d4cc534452ad50a87526626004.run_cpu_time_us, tablet-download-pool-ffacc2d4cc534452ad50a87526626004.run_cpu_time_us) (tablet-download-pool-ffacc2d4cc534452ad50a87526626004.run_wall_time_us, tablet-download-pool-ffacc2d4cc534452ad50a87526626004.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
*** Aborted at 1714702424 (unix time) try "date -d @1714702424" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000a75) received by PID 2677 (TID 0x7f9606038700) from PID 2677; stack trace: ***
@ 0x7f96b60e7980 (unknown) at ??:0
@ 0x7f96ab434fb7 gsignal at ??:0
@ 0x7f96ab436921 abort at ??:0
@ 0x7f96adee9dcd google::LogMessage::Fail() at ??:0
@ 0x7f96adeedb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f96adee97cc google::LogMessage::Flush() at ??:0
@ 0x7f96adeeaf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f96af566742 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f96af532f8b kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f96af52ccd6 kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f96be9f98de kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
I20240503 02:13:44.802120 4700 ts_tablet_manager.cc:902] T 89de6eace7d643e2a78385a67c771991 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 4001ccb96b064ad2a2ca9c61096516fe (127.2.157.67:34983)
@ 0x7f96bea11daf kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f96beb3c480 kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f96beb74a8d kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f96beb57981 kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
@ 0x7f96beb5770e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f96c39f998d std::function<>::operator()() at ??:0
@ 0x7f96af536b75 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f96af53a19c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f96af53a019 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f96c39f998d std::function<>::operator()() at ??:0
I20240503 02:13:44.852941 4713 ts_tablet_manager.cc:902] T a511b3bba2024783a999d7c7b616100e P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841)
@ 0x7f96af51501e kudu::Thread::SuperviseThread() at ??:0
I20240503 02:13:44.856061 4711 ts_tablet_manager.cc:902] T d43c2f4090f14b4baaf2a55a857ed245 P 5be54f33baf0422ab4ebf2f11e01e108: Initiating tablet copy from peer 6e0d1128f0404d609df72fd607c20650 (127.2.157.65:33841)
@ 0x7f96b60dc6db start_thread at ??:0
@ 0x7f96ab51771f clone at ??:0